HomeCurriculumSQL FundamentalsRelational Joins & Set Operations
Relational Joins & Set Operations

Relational Joins & Set Operations

Analyze physical join algorithms including Nested Loop, In-Memory Hash Join, Grace Hash Join, and Sort-Merge Join, alongside SQL set semantics and null-safe comparisons.

DT
Authored by DevThesis Systems Editorial Team
Reviewed for Technical Accuracy & Code Determinism

In relational database systems, normalization decomposes entity data across discrete tables to eliminate data redundancy and prevent update anomalies. Consequently, the primary computational challenge in relational query processing is re-assembling these decomposed tables into coherent analytical views. This reconstruction is achieved through Relational Joins and Set Operations.

This guide provides an exhaustive, university-grade examination of relational joins and set operations. We explore the formal relational algebra foundations of joins (Theta, Equi, Natural, Semi, and Anti joins), the physical execution algorithms implemented inside modern query optimizers (Nested Loop, Sort-Merge, and Grace Hash Joins), the subtle mathematical traps of ANSI Three-Valued Logic (3VL) and NULL semantics, and disk I/O cost estimation models.


1. Relational Join Theory & Codd's Foundations#

Formally, a relational join combines attributes from two relations into a single composite relation based on a join predicate.

text
       Relational Join Operator Hierarchy

       +-------------------------------------------------------------+
       |             Cartesian Product (Cross Join: R x S)           |
       |  Unconditional combination of all tuples: |R| * |S|         |
       +------------------------------+------------------------------+
                                      | (Add Conditional Predicate theta)
                                      v
       +-------------------------------------------------------------+
       |               Theta Join (R bowtie_theta S)                 |
       |  sigma_theta(R x S) - General comparison (<, <=, =, >=, >)  |
       +------------------------------+------------------------------+
                                      | (Restrict theta to Equality =)
                                      v
       +-------------------------------------------------------------+
       |                 Equi-Join (R bowtie_{A=B} S)                |
       |  Matches on explicit equality between attribute pairs       |
       +------------------------------+------------------------------+
                                      | (Project Single Common Key)
                                      v
       +-------------------------------------------------------------+
       |                 Natural Join (R bowtie S)                   |
       |  Implicit equi-join on all identically named attributes     |
       +-------------------------------------------------------------+

The Join Family Formalisms#

1. Theta Join (R ⨝_θ S)

A Theta Join is formally defined as a Cartesian product followed by a selection predicate θ:

R ⨝_θ S = σ_θ(R × S)Where θ is a propositional formula consisting of comparison operators (=, <, ≤, >, ≥, ≠).

2. Equi-Join

A specialized Theta Join where the predicate θ consists exclusively of equality comparisons:

R ⨝_{R.A = S.B} S = σ_{R.A = S.B}(R × S)3. Semijoin (R ⋉ S)

A Semijoin returns all tuples from relation R for which there exists at least one matching tuple in relation S, projecting only the attributes of R:

R ⋉_θ S = π_{attr(R)}(R ⨝_θ S)In SQL: Implemented via WHERE EXISTS (SELECT 1 FROM S WHERE ...) or IN (SELECT ...).

4. Antijoin (R ▷ S)

An Antijoin returns all tuples from relation R for which no matching tuple exists in relation S:

R ▷_θ S = R \ (R ⋉_θ S)In SQL: Implemented via WHERE NOT EXISTS (SELECT 1 FROM S WHERE ...) or LEFT JOIN S ON ... WHERE S.key IS NULL.


2. The SQL Join Taxonomy & Execution Semantics#

SQL-92 standardized explicit join syntax in the FROM clause, separating the join topology from row-level filtering in the WHERE clause.

text
+-----------------------------------------------------------------------------------------+
|                                SQL JOIN TAXONOMY MATRIX                                 |
+-----------------------------------------------------------------------------------------+
|  Join Syntax         |  Matching Criteria      |  Unmatched Left Rows |  Unmatched Right Rows|
+----------------------+-------------------------+----------------------+----------------------+
|  INNER JOIN          |  Predicate evaluates TRUE|  Discarded           |  Discarded           |
|  LEFT OUTER JOIN     |  Predicate evaluates TRUE|  Preserved (Padded NULL)| Discarded           |
|  RIGHT OUTER JOIN    |  Predicate evaluates TRUE|  Discarded           |  Preserved (Padded)  |
|  FULL OUTER JOIN     |  Predicate evaluates TRUE|  Preserved (Padded)  |  Preserved (Padded)  |
|  CROSS JOIN          |  Unconditional (All)    |  Preserved           |  Preserved           |
+-----------------------------------------------------------------------------------------+

1. `INNER JOIN`#

Returns only those tuples where the join condition evaluates strictly to TRUE:

sql
SELECT E.employee_id, E.name, D.department_name
FROM Employees AS E
INNER JOIN Departments AS D ON E.department_id = D.department_id;

If an employee has department_id = NULL, or a department has no employees, those records are discarded from the output.

2. `LEFT OUTER JOIN`#

Preserves every tuple from the left relation. If a left tuple finds no matching tuple in the right relation, the right attributes are populated with NULL:

sql
SELECT C.customer_id, C.name, O.order_id, O.total_amount
FROM Customers AS C
LEFT OUTER JOIN Orders AS O ON C.customer_id = O.customer_id;

3. The "Left Join to Inner Join" Query Bug#

A ubiquitous bug in enterprise analytics occurs when an engineer writes a LEFT JOIN and inadvertently converts it into an INNER JOIN by adding a filter on the right table in the WHERE clause:

sql
-- BUGGY QUERY: Converted to INNER JOIN!
SELECT C.customer_id, C.name, O.order_id, O.total_amount
FROM Customers AS C
LEFT JOIN Orders AS O ON C.customer_id = O.customer_id
WHERE O.order_status = 'COMPLETED';

Why this occurs:

For customers with no orders, the LEFT JOIN pads O.order_status with NULL. In the logical execution pipeline, Stage 2 (WHERE) evaluates after Stage 1 (FROM / JOIN). The predicate NULL = 'COMPLETED' evaluates to UNKNOWN (False), causing the WHERE clause to discard all customers who have no orders!

  • Correct Solution: Move the condition into the ON clause:

```sql

SELECT C.customer_id, C.name, O.order_id, O.total_amount

FROM Customers AS C

LEFT JOIN Orders AS O

ON C.customer_id = O.customer_id

AND O.order_status = 'COMPLETED';

```


3. Database Engine Physical Join Algorithms#

Relational query optimizers (Cost-Based Optimizers / CBO) evaluate physical execution paths to join two tables. Modern engines implement three primary physical join algorithms:

text
       Physical Join Engine Execution Mechanics

       1. Nested Loop Join (NLJ)
          For each row r in Outer_Table:
              Scan Inner_Table (or B+ Tree Index) for matching keys.
          [ Best for: Small outer table with indexed inner table ]

       2. Sort-Merge Join (SMJ)
          Sort Outer_Table on Join_Key.
          Sort Inner_Table on Join_Key.
          Scan both sorted streams simultaneously with dual pointers.
          [ Best for: Pre-sorted data, range predicates (<, <=) ]

       3. Hash Join (HJ)
          Phase 1 (Build): Hash smaller table into in-memory Hash Table.
          Phase 2 (Probe): Stream larger table, probing hash buckets.
          [ Best for: Large unsorted tables with equi-joins (=) ]

1. Nested Loop Join (NLJ)#

  • Simple Nested Loop Join: Iterates through the entire inner table for every row of the outer table (O(|R| · |S|)).
  • Index Nested Loop Join (INLJ): For each row in the outer table R, the engine uses a B+ tree index on the inner table S to find matches.

Time Complexity = O(|R| · log |S|)Optimal when the outer table is small (e.g., filtered to 10 rows) and the inner table has an index on the join key.

2. Sort-Merge Join (SMJ)#

  • Both tables are sorted by the join attribute.
  • Two pointers advance through the sorted streams in linear time.

Time Complexity = O(|R| log |R| + |S| log |S| + |R| + |S|)Optimal when an existing clustered index provides pre-sorted order, or when physical memory cannot support a hash table.

3. Grace Hash Join (HJ)#

Designed for massive datasets that exceed available memory (work_mem):

  • Phase 1 (Partitioning): Both relations R and S are partitioned into k buckets on disk using hash function h_1(key) mod k.
  • Phase 2 (Build & Probe): Each bucket pair (R_i, S_i) is processed independently in memory using a second hash function h_2(key).

Total I/O Cost = 3 × (Pages(R) + Pages(S))Hash joins require an equality predicate (=); they cannot execute non-equi joins (<, >).


4. Set Operations: Relational Set Theory#

Set operations combine the result sets of two independent queries:

text
+-----------------------------------------------------------------------------------------+
|                                SQL SET OPERATIONS MATRIX                                |
+-----------------------------------------------------------------------------------------+
|  Set Operation       |  Mathematical Meaning   |  Deduplication Behavior |  Performance |
+----------------------+-------------------------+-------------------------+--------------+
|  UNION               |  Set Union (R union S)  |  DEDUPLICATES (Unique)  |  Slow (Sort) |
|  UNION ALL           |  Multiset Bag Union     |  PRESERVES DUPLICATES   |  Fast (Stream|
|  INTERSECT           |  Set Intersection       |  DEDUPLICATES           |  Moderate    |
|  EXCEPT / MINUS      |  Set Difference (R \ S) |  DEDUPLICATES           |  Moderate    |
+-----------------------------------------------------------------------------------------+

`UNION` vs. `UNION ALL` Performance Trade-Off#

  • `UNION ALL`: Concatenates the two output streams directly. It requires zero sorting, zero hashing, and zero deduplication buffers (O(|R| + |S|)).
  • `UNION`: After concatenating, the database must sort the entire combined relation or insert all rows into a hash table to eliminate duplicate rows (O((|R| + |S|) log (|R| + |S|))).

Engineering Best Practice: Always use UNION ALL unless deduplication is an explicit business requirement.


5. Three-Valued Logic (3VL) & The `NULL` Trap#

ANSI SQL operates on Three-Valued Logic (3VL) comprising: TRUE, FALSE, and UNKNOWN. A missing or unknown value is represented as NULL.

text
       Three-Valued Truth Tables (AND, OR, NOT)

       AND      | TRUE    FALSE   UNKNOWN        OR       | TRUE    FALSE   UNKNOWN
       ---------+------------------------        ---------+------------------------
       TRUE     | TRUE    FALSE   UNKNOWN        TRUE     | TRUE    TRUE    TRUE
       FALSE    | FALSE   FALSE   FALSE          FALSE    | TRUE    FALSE   UNKNOWN
       UNKNOWN  | UNKNOWN FALSE   UNKNOWN        UNKNOWN  | TRUE    UNKNOWN UNKNOWN

       NOT(TRUE) = FALSE,   NOT(FALSE) = TRUE,   NOT(UNKNOWN) = UNKNOWN

The Catastrophic `NOT IN (subquery)` NULL Trap#

Consider checking which customers have never placed an order:

sql
-- DANGEROUS SQL ANTI-PATTERN: The NOT IN NULL Trap
SELECT customer_id, name 
FROM Customers 
WHERE customer_id NOT IN (SELECT customer_id FROM Orders);

What happens if a single row in `Orders` has `customer_id = NULL`?

The query evaluates:

customer_id NOT IN (101, 102, NULL)Which expands logically to:

(customer_id ≠ 101) AND (customer_id ≠ 102) AND (customer_id ≠ NULL)Because any equality or inequality comparison with NULL evaluates to UNKNOWN:

Condition = TRUE AND TRUE AND UNKNOWN ──> UNKNOWNIn a SQL WHERE clause, rows are returned only if the predicate evaluates strictly to `TRUE`!

Because the condition evaluates to UNKNOWN for every single customer in the database, the query returns ZERO ROWS! The presence of a single NULL in the subquery completely destroys the query result.

The Solution: `NOT EXISTS`:

sql
-- SAFE & OPTIMAL: Using NOT EXISTS
SELECT C.customer_id, C.name
FROM Customers AS C
WHERE NOT EXISTS (
    SELECT 1 FROM Orders AS O 
    WHERE O.customer_id = C.customer_id
);

EXISTS evaluates whether the subquery returns at least one tuple. If O.customer_id is null, the join predicate O.customer_id = C.customer_id evaluates to UNKNOWN, the subquery returns zero rows, and NOT EXISTS evaluates cleanly to TRUE.


6. Complete Python Implementation: Physical Join Algorithms#

The following script implements the three primary physical database join algorithms (Index Nested Loop, Sort-Merge, and Hash Join) in pure Python, benchmarking execution complexity:

python
"""
Physical Database Join Algorithms Implementation
Demonstrating Nested Loop, Sort-Merge, and In-Memory Hash Joins
"""

from typing import List, Dict, Any

class JoinEngine:
    @staticmethod
    def nested_loop_join(outer: List[Dict[str, Any]], inner: List[Dict[str, Any]], key: str) -> List[Dict[str, Any]]:
        """
        Block Nested Loop Join: O(|R| * |S|)
        """
        result = []
        for r in outer:
            for s in inner:
                if r.get(key) == s.get(key) and r.get(key) is not None:
                    # Merge row dictionaries
                    result.append({**r, **{f"inner_{k}": v for k, v in s.items()}})
        return result

    @staticmethod
    def sort_merge_join(outer: List[Dict[str, Any]], inner: List[Dict[str, Any]], key: str) -> List[Dict[str, Any]]:
        """
        Sort-Merge Join: O(|R| log |R| + |S| log |S| + |R| + |S|)
        """
        # Step 1: Sort both streams
        sorted_outer = sorted([r for r in outer if r.get(key) is not None], key=lambda x: x[key])
        sorted_inner = sorted([s for s in inner if s.get(key) is not None], key=lambda x: x[key])

        result = []
        i, j = 0, 0
        n, m = len(sorted_outer), len(sorted_inner)

        # Step 2: Linear dual-pointer merge scan
        while i < n and j < m:
            val_r = sorted_outer[i][key]
            val_s = sorted_inner[j][key]

            if val_r == val_s:
                # Handle duplicate keys on inner stream
                k = j
                while k < m and sorted_inner[k][key] == val_r:
                    result.append({**sorted_outer[i], **{f"inner_{col}": v for col, v in sorted_inner[k].items()}})
                    k += 1
                i += 1
            elif val_r < val_s:
                i += 1
            else:
                j += 1

        return result

    @staticmethod
    def hash_join(outer: List[Dict[str, Any]], inner: List[Dict[str, Any]], key: str) -> List[Dict[str, Any]]:
        """
        In-Memory Hash Join: O(|R| + |S|)
        """
        result = []
        # Phase 1: Build Hash Table on inner relation
        hash_table: Dict[Any, List[Dict[str, Any]]] = {}
        for s in inner:
            val = s.get(key)
            if val is not None:
                if val not in hash_table:
                    hash_table[val] = []
                hash_table[val].append(s)

        # Phase 2: Probe Hash Table with outer relation
        for r in outer:
            val = r.get(key)
            if val is not None and val in hash_table:
                for match in hash_table[val]:
                    result.append({**r, **{f"inner_{k}": v for k, v in match.items()}})

        return result


# Verification execution
if __name__ == '__main__':
    customers = [
        {'cust_id': 1, 'name': 'Alice'},
        {'cust_id': 2, 'name': 'Bob'},
        {'cust_id': 3, 'name': 'Charlie'},
    ]
    orders = [
        {'order_id': 101, 'cust_id': 1, 'amount': 250.0},
        {'order_id': 102, 'cust_id': 1, 'amount': 80.0},
        {'order_id': 103, 'cust_id': 3, 'amount': 500.0},
        {'order_id': 104, 'cust_id': 9, 'amount': 120.0},
    ]

    hj_res = JoinEngine.hash_join(customers, orders, 'cust_id')
    smj_res = JoinEngine.sort_merge_join(customers, orders, 'cust_id')

    print(f"[*] Hash Join Matches Found: {len(hj_res)}")
    print(f"[*] Sort-Merge Matches Found: {len(smj_res)}")

7. Mathematical Foundations of Join Operations#

1. Disk Page I/O Cost Model Formulation#

In database query optimization, performance is modeled by counting secondary storage Disk Page Transfers.

Let:

  • B(R): Number of physical disk blocks (pages) storing relation R.
  • B(S): Number of physical disk blocks (pages) storing relation S.
  • |R|: Number of tuples in R.
  • |S|: Number of tuples in S.
  • M: Number of available memory buffer pages.

1. Block Nested Loop Join (BNLJ) Cost

The engine loads M - 2 buffer pages of the outer table R into memory, allocates 1 page for the inner table S, and 1 page for output.

Cost_{BNLJ} = B(R) + ≤ft⌈ (B(R)) / (M - 2) ⌉ × B(S)2. Sort-Merge Join (SMJ) Cost

Assuming relations are not pre-sorted. Sorting via external two-pass merge sort costs 2 · B log_M B passes:

Cost_{Sort} = 2 · B(R) · ⌈ log_{M-1}(B(R)) ⌉ + 2 · B(S) · ⌈ log_{M-1}(B(S)) ⌉Cost_{Merge} = B(R) + B(S)Total Cost_{SMJ} ≈ 3 · (B(R) + B(S))3. Grace Hash Join Cost

  • Partitioning pass: Read and write both relations to partitioned buckets on disk: 2 · (B(R) + B(S)).
  • Probe pass: Read all partitions back into memory: B(R) + B(S).

Total Cost_{Hash} = 3 · (B(R) + B(S))---

8. Common Pitfalls & Join Traps#

text
+-----------------------------------------------------------------------------------------+
|                                SQL JOIN PITFALL MATRIX                                  |
+-----------------------------------------------------------------------------------------+
|  Join Anti-Pattern              |  Root Cause             |  Optimal Engineering Fix    |
+---------------------------------+-------------------------+-----------------------------+
|  Accidental Cartesian Explosion |  Missing or incomplete  |  Enforce join conditions on |
|                                 |  join ON predicate      |  composite primary keys     |
+---------------------------------+-------------------------+-----------------------------+
|  WHERE clause on LEFT JOIN      |  Filter on right table  |  Move filter into the       |
|                                 |  converts to INNER JOIN |  ON clause of the LEFT JOIN |
+---------------------------------+-------------------------+-----------------------------+
|  NOT IN with NULL subquery      |  Three-Valued Logic     |  Always use NOT EXISTS or   |
|                                 |  evaluates to UNKNOWN   |  LEFT JOIN ... IS NULL      |
+---------------------------------+-------------------------+-----------------------------+

9. Checkpoint Practice Problems & Step-by-Step Solutions#

Problem 1: Disk Page I/O Cost Comparison: Nested Loop vs. Hash Join#

Scenario: A database query optimizer evaluates joining two large tables:

  • Relation R (Orders): B(R) = 10,000 pages.
  • Relation S (Customers): B(S) = 2,000 pages.

The query buffer pool allocates M = 52 buffer pages to the join. Assume no indexes exist on the join keys.

Questions:

  1. Calculate the total Disk Page I/O cost using a Block Nested Loop Join (BNLJ) with S as the outer relation.
  2. Calculate the total Disk Page I/O cost using a Block Nested Loop Join (BNLJ) with R as the outer relation. Why does the choice of outer table matter?
  3. Calculate the total Disk Page I/O cost using a Grace Hash Join.

Solution:

Step 1: BNLJ Cost with S (Smaller Table) as Outer Relation:

Outer relation: S, B(S) = 2,000.

Inner relation: R, B(R) = 10,000.

Buffer memory: M = 52 ──> M - 2 = 50 pages.

Chunks of Outer = ≤ft⌈ (B(S)) / (M - 2) ⌉ = ≤ft⌈ (2,000) / (50) ⌉ = 40 chunksCost = B(S) + (Chunks × B(R))Cost = 2,000 + (40 × 10,000) = 2,000 + 400,000 = 402,000 Page I/OsStep 2: BNLJ Cost with R (Larger Table) as Outer Relation:

Outer relation: R, B(R) = 10,000.

Inner relation: S, B(S) = 2,000.

Chunks of Outer = ≤ft⌈ (10,000) / (50) ⌉ = 200 chunksCost = B(R) + (Chunks × B(S))Cost = 10,000 + (200 × 2,000) = 10,000 + 400,000 = 410,000 Page I/Os- Analysis: Placing the smaller table as the outer relation saves 8,000 disk page reads, because the initial scan cost of the outer table (B(S) = 2,000) is lower than B(R) = 10,000.

Step 3: Grace Hash Join Cost:

Under a 2-pass Grace Hash Join:

Cost_{Hash} = 3 · (B(R) + B(S))Cost_{Hash} = 3 · (10,000 + 2,000) = 3 · 12,000 = 36,000 Page I/OsComparing 36,000 I/Os against 402,000 I/Os: the Hash Join achieves an 11.16x performance speedup, proving why cost-based optimizers select hash joins for large unindexed tables.


Problem 2: Three-Valued Logic & The `NOT IN` vs. `NOT EXISTS` Evaluation#

Scenario: A corporate database holds two tables:

  • Employees:

- (1, 'Alice')

- (2, 'Bob')

- (3, 'Charlie')

  • Projects:

- (P1, 1) (Assigned to Alice)

- (P2, NULL) (Unassigned project)

An HR manager runs Query A:

sql
SELECT name FROM Employees WHERE employee_id NOT IN (SELECT employee_id FROM Projects);

A senior database architect runs Query B:

sql
SELECT name FROM Employees WHERE NOT EXISTS (SELECT 1 FROM Projects WHERE Projects.employee_id = Employees.employee_id);

Questions:

  1. Detail the step-by-step three-valued logic evaluation of Query A for Alice (employee_id = 1) and Bob (employee_id = 2). What rows does Query A return?
  2. Detail the step-by-step evaluation of Query B for Alice and Bob. What rows does Query B return?

Solution:

Step 1: Evaluate Query A (`NOT IN`):

The subquery (SELECT employee_id FROM Projects) produces the multiset {1, NULL}.

  • For Alice (`employee_id = 1`):

Evaluate: 1 NOT IN (1, NULL)

≡ NOT(1 = 1 OR 1 = NULL)1 = 1 ──> TRUE1 = NULL ──> UNKNOWNTRUE OR UNKNOWN ──> TRUENOT(TRUE) ──> FALSEAlice is excluded.

  • For Bob (`employee_id = 2`):

Evaluate: 2 NOT IN (1, NULL)

≡ NOT(2 = 1 OR 2 = NULL)2 = 1 ──> FALSE2 = NULL ──> UNKNOWNFALSE OR UNKNOWN ──> UNKNOWNNOT(UNKNOWN) ──> UNKNOWNUnder 3VL, the WHERE clause accepts only TRUE. Because UNKNOWN is not TRUE, Bob is excluded!

Query A returns: EMPTY SET (Zero rows!).

Step 2: Evaluate Query B (`NOT EXISTS`):

  • For Alice (`employee_id = 1`):

The correlated subquery checks WHERE Projects.employee_id = 1.

Match found: Project P1 (1 = 1 is TRUE).

The subquery returns 1 row.

NOT EXISTS (1 row) evaluates to `FALSE`. Alice is excluded.

  • For Bob (`employee_id = 2`):

The subquery checks WHERE Projects.employee_id = 2.

- For Project P1: 1 = 2 is FALSE.

- For Project P2: NULL = 2 is UNKNOWN.

Zero rows satisfy the subquery condition. The subquery returns an empty set.

NOT EXISTS (empty set) evaluates to `TRUE`!

Bob is included!

Query B returns: `Bob`.

NOT EXISTS correctly handles nulls, preventing silent query failure.


Problem 3: Multi-Table Star Schema Join Selectivity & Cardinality Estimation#

Scenario: A data warehouse processes a star schema join query:

sql
SELECT COUNT(*)
FROM FactSales AS F
JOIN DimCustomer AS C ON F.customer_id = C.customer_id
JOIN DimStore AS S ON F.store_id = S.store_id
WHERE C.region = 'Europe' AND S.store_type = 'Flagship';

Database statistics report:

  • |F| = 50,000,000 sales rows.
  • |C| = 1,000,000 customer rows. Selectivity of C.region = 'Europe': σ_C = 0.20.
  • |S| = 500 store rows. Selectivity of S.store_type = 'Flagship': σ_S = 0.10.
  • Every row in FactSales references a valid customer_id and store_id (Uniform foreign key distribution).

Questions:

  1. Calculate the number of filtered customer rows and filtered store rows.
  2. Assuming independent uniform attribute distributions, calculate the estimated output cardinality of the complete query.

Solution:

Step 1: Calculate Filtered Dimension Cardinalities:

  • Filtered Customers:

|C_{filtered}| = |C| × σ_C = 1,000,000 × 0.20 = 200,000 customers- Filtered Stores:

|S_{filtered}| = |S| × σ_S = 500 × 0.10 = 50 storesStep 2: Calculate Expected Join Cardinality:

Under foreign key referential integrity with uniform distribution:

  • The probability that a random fact record matches a European customer is:

P(Customer Match) = (|C_{filtered}|) / (|C|) = σ_C = 0.20- The probability that a random fact record matches a Flagship store is:

P(Store Match) = (|S_{filtered}|) / (|S|) = σ_S = 0.10Assuming independence:

Combined Selectivity σ_{total} = σ_C × σ_S = 0.20 × 0.10 = 0.02Estimated output cardinality:

Estimated Rows = |F| × σ_{total} = 50,000,000 × 0.02 = 1,000,000 rows---

Problem 4: Self-Join Hierarchical Employee-Manager Path Resolution#

Scenario: An organization stores its management hierarchy within a single self-referencing table:

sql
CREATE TABLE staff (
    emp_id INT PRIMARY KEY,
    name VARCHAR(50) NOT NULL,
    manager_id INT NULL,
    salary DECIMAL(10, 2) NOT NULL
);

Data records:

  • (1, 'CEO_Sarah', NULL, 250000)
  • (2, 'VP_Alex', 1, 150000)
  • (3, 'VP_Dan', 1, 160000)
  • (4, 'Dev_Eve', 2, 100000)
  • (5, 'Dev_Frank', 2, 95000)

Questions:

  1. Write a SQL query using a Self-Join that returns the employee's name, their direct manager's name, and the salary difference (manager_salary - employee_salary).
  2. If CEO_Sarah has manager_id = NULL, what type of join must be used to ensure Sarah is not omitted from the organizational report?

Solution:

Step 1: Formulate the Self-Join Query:

To compare employees against their direct managers, the staff table must be referenced twice under distinct aliases (E for Employee, M for Manager):

sql
SELECT 
    E.name AS employee_name,
    M.name AS manager_name,
    (M.salary - E.salary) AS salary_differential
FROM staff AS E
LEFT OUTER JOIN staff AS M ON E.manager_id = M.emp_id;

Step 2: Selection of Join Type:

If an INNER JOIN were used:

Because CEO_Sarah has manager_id = NULL, the condition E.manager_id = M.emp_id (NULL = 1) evaluates to UNKNOWN. Sarah would be completely excluded from the report.

A `LEFT OUTER JOIN` must be used, ensuring the top-level executive is preserved with manager_name = NULL and salary_differential = NULL.

Key Architectural Takeaways

  • Relational joins are Cartesian products filtered by join predicates: R ⋈_θ S = σ_θ(R × S).
  • Hash Join divides execution into an in-memory build phase on the smaller relation and a streaming probe phase on the larger relation.
  • Sort-Merge Join provides O(1) auxiliary space and optimal performance when inputs are pre-sorted by clustered indexes.
  • UNION ALL preserves duplicate rows without sorting, providing superior throughput compared to deduplicating UNION.

Hands-on Code Playground Available

Execute, edit, and experiment with code for this topic in the live sandbox.

Practice Check • Quick Quiz

Test Your Understanding

Under what condition does an In-Memory Hash Join spill temporary partitions to disk (Grace Hash Join)?

A
When the query contains an ORDER BY clause
B
When the build relation's hash table exceeds the configured working memory (e.g. work_mem)
C
When the inner relation has a foreign key constraint
D
When the query uses UNION ALL instead of UNION