SQL’s Hidden Gem: Exploring WITH RECURSIVE for Hierarchical and Recursive Queries

What Is WITH RECURSIVE?#

WITH RECURSIVE is an extension of SQL’s Common Table Expression (CTE) that allows queries to loop back on their own results. It’s a game-changer for handling hierarchical or recursive data, such as organizational charts, file systems, or finding paths in a network. Think of it as SQL’s way of saying, “Let me build this step by step for you.”

It works in two parts:

1. Base Case: The starting point, where you define the initial data.
2. Recursive Case: The part that builds on previous results, adding new data until the process is complete.

These parts are connected with UNION ALL (or occasionally UNION), and the recursion stops when there’s no more data to process. Let’s bring this to life with some practical examples.

Example 1: Mapping a Company Hierarchy#

Imagine you’re working with a company’s employee structure, where everyone reports to someone except the CEO. Let’s use WITH RECURSIVE to map out the entire hierarchy.

Table Structure#

CREATE TABLE employees (
    employee_id INT PRIMARY KEY,
    name VARCHAR(100),
    manager_id INT
);

Sample Data#

INSERT INTO employees (employee_id, name, manager_id) VALUES
(1, 'Sarah', NULL), -- The CEO
(2, 'John', 1),
(3, 'Emma', 1),
(4, 'Mike', 2),
(5, 'Lisa', 2),
(6, 'Tom', 3);

Query#

WITH RECURSIVE hierarchy AS (
    -- Base case: Start with the CEO
    SELECT employee_id, name, manager_id
    FROM employees
    WHERE manager_id IS NULL
    UNION ALL
    -- Recursive case: Find employees reporting to others
    SELECT e.employee_id, e.name, e.manager_id
    FROM employees e
    INNER JOIN hierarchy h ON e.manager_id = h.employee_id
)
SELECT * FROM hierarchy;

What’s Happening?#

This query starts with Sarah, the CEO (who has no manager), and works its way down the chain. It lists John and Emma reporting to Sarah, Mike and Lisa under John, and Tom under Emma. It’s like sketching out the company’s org chart with a few lines of SQL!

Example 2: Generating a Number Sequence#

Sometimes, you just need something simple, like generating numbers from 1 to 10. With WITH RECURSIVE, this is a breeze—no loops required.

Query#

WITH RECURSIVE numbers AS (
    SELECT 1 AS n
    UNION ALL
    SELECT n + 1
    FROM numbers
    WHERE n < 10
)
SELECT * FROM numbers;

Output#

n
1
2
3
...
10

It’s like SQL saying, “Need a quick number list? I’ve got you covered!” Simple, elegant, and super useful.

Example 3: Finding Routes from New York to Los Angeles#

Let’s get a bit more adventurous. Suppose you have a table of roads between cities, and you want to find all possible routes from New York to Los Angeles, along with their total distances. WITH RECURSIVE is here to help.

Table Structure#

CREATE TABLE roads (
    city_from VARCHAR(50),
    city_to VARCHAR(50),
    distance INT
);

Sample Data#

INSERT INTO roads (city_from, city_to, distance) VALUES
('New York', 'Chicago', 800),
('Chicago', 'Denver', 1000),
('New York', 'Boston', 200),
('Boston', 'Los Angeles', 3000),
('Denver', 'Los Angeles', 1100);

Query#

WITH RECURSIVE paths AS (
    -- Base case: Start from New York
    SELECT city_from, city_to, distance, ARRAY[city_from, city_to] AS path
    FROM roads
    WHERE city_from = 'New York'
    UNION ALL
    -- Recursive case: Add more routes
    SELECT r.city_from, r.city_to, p.distance + r.distance, p.path || r.city_to
    FROM roads r
    INNER JOIN paths p ON r.city_from = p.city_to
    WHERE NOT r.city_to = ANY(p.path) -- Prevent loops
)
SELECT * FROM paths
WHERE city_to = 'Los Angeles';

Output#

This query lists all possible routes from New York to Los Angeles, along with their distances. For example:

• New York → Boston → Los Angeles: 3200 miles
• New York → Chicago → Denver → Los Angeles: 2900 miles

It’s like having your own route planner, powered by SQL!

Why Is WITH RECURSIVE So Useful?#

WITH RECURSIVE is a versatile tool that shines in several scenarios:

• Hierarchies: Perfect for organizational charts, file systems, or category trees.
• Graph Analysis: Great for finding paths, like road networks or dependency graphs.
• Clean Code: It simplifies complex logic, saving you from writing convoluted loops.

Tips for Success#

• Prevent Infinite Loops: Craft your WHERE conditions carefully to avoid endless recursion.
• Choose UNION ALL: It’s faster than UNION since it doesn’t check for duplicates, making your queries zippy.
• Check Database Support: WITH RECURSIVE works in modern databases like PostgreSQL, MySQL 8.0+, Oracle, and SQL Server. Double-check if you’re using an older system.

Final Thoughts#

WITH RECURSIVE is like a trusty companion for navigating complex data in SQL. Whether you’re mapping out a company’s hierarchy, generating a sequence, or planning a cross-country trip, it makes the process smooth and intuitive. So, fire up your database and give it a try! What exciting project will you tackle with WITH RECURSIVE next?