Memory Profiling, C-Extensions & Performance

Python is celebrated for its expressiveness, developer ergonomics, and rapid development cadences. However, because CPython is an interpreted, dynamically typed language where every integer, string, and function is a heap-allocated object, naive Python implementations can incur substantial memory bloat and execution latency.

Building large-scale, high-density data pipelines, high-frequency trading services, and production AI backends in Python requires mastering CPython's low-level memory architecture: the pymalloc arena-pool-block hierarchy, generational cyclic garbage collection, memory profiling tools (tracemalloc), memory compression via __slots__, and native C/Rust extensions that release the Global Interpreter Lock.

This guide provides an exhaustive, university-grade architectural analysis of Python memory management, profiling diagnostics, and native extension engineering. We dissect the internal mechanics of CPython's memory allocator, formulate fragmentation and generational garbage collection dynamics, evaluate C foreign function interfaces (ctypes, CFFI, Cython, PyO3), optimize data structures for massive memory savings, and walk through four comprehensive quantitative checkpoint problems.

1. CPython Memory Management Architecture#

CPython abstracts system memory through a sophisticated four-tiered hierarchical allocator designed to balance allocation speed, cache locality, and operating system heap fragmentation.

+-------------------------------------------------------------------------------------------------+
|                                 CPYTHON MEMORY ALLOCATION HIERARCHY                             |
|                                                                                                 |
|   Layer 3: Object-Specific Allocators (Free Lists for PyLongObject, PyFloatObject, etc.)        |
|   ===========================================================================================   |
|   Layer 2: Python Small Object Allocator (pymalloc)                                             |
|            - Manages allocations <= 512 bytes.                                                  |
|            - Subdivides memory into Arenas (256 KB) -> Pools (4 KB) -> Blocks (8-512 bytes).    |
|   ===========================================================================================   |
|   Layer 1: C Standard Allocator (PyMem_RawMalloc / malloc / calloc / free)                      |
|            - Manages large objects > 512 bytes directly via OS heap.                            |
|   ===========================================================================================   |
|   Layer 0: Operating System Virtual Memory Manager (OS Virtual Memory Pages / mmap / brk)       |
+-------------------------------------------------------------------------------------------------+

The `pymalloc` Small Object Allocator#

When an application creates thousands of small objects (e.g., integers, short strings, small tuples, dictionaries), invoking the operating system's malloc() for every object causes catastrophic heap fragmentation and CPU kernel context-switch overhead. To eliminate this, CPython routes all memory requests of 512 bytes or smaller through pymalloc:

1. Arenas (256 KB):

An arena is a 256-kilobyte contiguous memory chunk requested from the OS via malloc() or mmap(). Arenas are aligned on 256 KB memory boundaries.

1. Pools (4 KB):

Each arena is subdivided into 64 distinct Pools of 4 kilobytes each (matching the physical CPU memory page size). A pool is dedicated to objects of a single, uniform size class.

1. Blocks (8 to 512 bytes):

Each pool is partitioned into equal-sized blocks. pymalloc establishes 64 distinct size classes in increments of 8 bytes:

- Size Class 0: 8-byte blocks

- Size Class 1: 16-byte blocks

- Size Class 2: 24-byte blocks

- ...

- Size Class 63: 512-byte blocks

When an object of size S is allocated (where S ≤ 512), pymalloc rounds S up to the nearest multiple of 8, finds an available pool matching that size class, and returns a pointer to a free block in O(1) time.

Reference Counting & The Generational Garbage Collector#

CPython combines two distinct garbage collection algorithms:

1. Deterministic Reference Counting#

Every Python object header (PyObject) contains an internal counter: ob_refcnt.

• When an object reference is copied or assigned, ob_refcnt is incremented (Py_INCREF).
• When a reference leaves scope or is reassigned, ob_refcnt is decremented (Py_DECREF).
• As soon as ob_refcnt drops to zero, the object is immediately deallocated and its memory block returned to its pool.

2. The Cyclic Garbage Collector#

Reference counting alone cannot reclaim cyclic references (where Object A points to Object B, and Object B points to Object A):

# A Cyclic Reference Leak (Uncollectable by pure reference counting!)
class Node:
    def __init__(self):
        self.partner = None

a = Node()
b = Node()
a.partner = b
b.partner = a
del a
del b
# Ref counts of a and b remain at 1! They are unreachable, but never reach 0.

To detect and reclaim cyclic references, CPython implements a Generational Garbage Collector (Modules/gcmodule.c):

• All container objects (objects capable of holding references, such as dict, list, tuple, set, and custom class instances) are linked into doubly-linked ring buffers.
• The GC organizes objects into three generations:

- Generation 0: Newly allocated container objects. Collected frequently.

- Generation 1: Objects surviving one Generation 0 collection. Collected moderately.

- Generation 2: Long-lived objects (surviving multiple GC cycles). Collected infrequently.

The Trial Decrement Algorithm#

During a collection cycle, the GC performs trial decrements:

1. It copies each container's reference count into an internal field gc_refs.
2. For each container, it traverses all outgoing references and decrements the target's gc_refs by 1.
3. After the traversal, any container whose gc_refs == 0 was referenced only from within the cyclic group! These isolated objects are unreachable from external roots and are safely deallocated.

2. Memory Profiling and Leak Detection Tooling#

Diagnosing memory bloat in production environments requires specialized introspection tooling.

Shallow vs. Deep Memory: The `sys.getsizeof` Fallacy#

The built-in sys.getsizeof(obj) function returns only the memory allocated for the outer container object itself, ignoring the memory of referenced items:

import sys

# A list of 1,000,000 integer objects
data = [i for i in range(1_000_000)]

# MISLEADING: Returns only the memory of the list pointer array (approx. 8 MB)
print("Shallow size:", sys.getsizeof(data)) # ~8,000,056 bytes

# TRUE DEEP SIZE: Includes the 1,000,000 PyLongObject instances (approx. 36 MB total!)

To compute deep memory consumption, developers use recursive traversal algorithms or memory profiling libraries.

Production Diagnostics with `tracemalloc`#

The standard library tracemalloc module tracks memory allocations at the exact Python source code line where memory was requested:

import tracemalloc

# Start tracking memory allocations with 10 stack frames of context
tracemalloc.start(10)

# Execute suspected memory-leaking workload
snapshot1 = tracemalloc.take_snapshot()
leaky_workload()
snapshot2 = tracemalloc.take_snapshot()

# Compare snapshots to identify leaking lines
stats = snapshot2.compare_to(snapshot1, 'lineno')
for stat in stats[:5]:
    print(stat)

Memory Compression via `__slots__`#

By default, every instance of a Python class maintains an internal dictionary __dict__ to allow dynamic attribute assignment at runtime. In CPython, a dictionary incurs an overhead of 100 to 200 bytes per instance.

When instantiating millions of objects (such as database rows or coordinate points), declaring __slots__ replaces __dict__ with a compact, fixed-size C array of pointers:

class StandardPoint:
    def __init__(self, x, y):
        self.x = x
        self.y = y

class SlottedPoint:
    __slots__ = ('x', 'y')  # Replaces __dict__ with a C-level fixed struct array
    def __init__(self, x, y):
        self.x = x
        self.y = y

Memory reduction:

• StandardPoint: Instance header (16 bytes) + __dict__ pointer (8 bytes) + PyDictObject (152 bytes) + values ≈ 152 bytes per instance.
• SlottedPoint: Instance header (16 bytes) + two C-level PyObject* pointers (16 bytes) ≈ 48 bytes per instance.
• Memory Savings: Approximately 68.4% memory reduction across millions of records.

3. High-Performance Native Extensions: C, CFFI, Cython & PyO3#

When Python code hits CPU bottlenecks that cannot be resolved through vectorization, engineers interface with compiled native code.

+-------------------------------------------------------------------------------------------------+
|                                 PYTHON NATIVE EXTENSION OPTIONS                                 |
+-------------------+-----------------------+-----------------------+-----------------------------+
| Framework         | Technology / Language | Compilation Step      | GIL Management              |
+-------------------+-----------------------+-----------------------+-----------------------------+
| CPython C API     | Raw C / C++           | Required (.so / .dll) | Manual:                     |
|                   |                       |                       | Py_BEGIN_ALLOW_THREADS      |
+-------------------+-----------------------+-----------------------+-----------------------------+
| ctypes            | Pure Python           | None (Loads existing  | Automatic for foreign calls |
|                   | Dynamic FFI           | shared libraries)     |                             |
+-------------------+-----------------------+-----------------------+-----------------------------+
| CFFI              | C Foreign Function    | Optional (ABI mode vs | Native C declarations;      |
|                   | Interface             | API mode)             | Releases GIL cleanly        |
+-------------------+-----------------------+-----------------------+-----------------------------+
| Cython            | Python/C Superset     | Compiles .pyx to C    | with nogil: blocks          |
+-------------------+-----------------------+-----------------------+-----------------------------+
| PyO3              | Rust                  | Compiles Rust to      | py.allow_threads(...)       |
|                   |                       | native Python wheels  | Memory-safe by design       |
+-------------------+-----------------------+-----------------------+-----------------------------+

Releasing the GIL in Native C Extensions#

In pure Python, CPU-bound computations are serialized by the Global Interpreter Lock. However, inside a C extension, if the native code is performing pure mathematical computations on raw C buffers without manipulating Python object references (PyObject*), it can release the GIL, enabling true multi-core parallel scaling:

// Native C Function executing in parallel across OS threads
static PyObject* native_parallel_hash(PyObject* self, PyObject* args) {
    const char* input_buffer;
    Py_ssize_t buffer_length;

    if (!PyArg_ParseTuple(args, "s#", &input_buffer, &buffer_length)) {
        return NULL;
    }

    // RELEASE THE GIL: Other Python threads can now execute concurrently!
    Py_BEGIN_ALLOW_THREADS

    // Pure C computation on raw memory buffer (No PyObject operations permitted!)
    compute_sha256_heavy_c(input_buffer, buffer_length);

    // RE-ACQUIRE THE GIL: Must re-acquire lock before creating Python return object
    Py_END_ALLOW_THREADS

    return PyLong_FromLong(1);
}

4. Production Engineering Case Study: High-Density Analytics Microservice Memory Optimization#

Consider a financial telemetry service storing 10,000,000 equity quote records in memory. The service was crashing with Out-Of-Memory (OOM) errors in a 4 GB Docker container.

Step 1: Profiling the Baseline Architecture#

# Initial Naive Implementation
class Quote:
    def __init__(self, symbol: str, bid: float, ask: float, volume: int):
        self.symbol = symbol
        self.bid = bid
        self.ask = ask
        self.volume = volume

Memory Audit:

• 10,000,000 instances of Quote.
• Measured resident memory via tracemalloc: 1.52 Gigabytes.
• Additional overhead from dictionary resize buckets and reference fragmentation.

Step 2: Applying `__slots__` and Flyweight String Interning#

import sys

class OptimizedQuote:
    __slots__ = ('symbol', 'bid', 'ask', 'volume')
    
    def __init__(self, symbol: str, bid: float, ask: float, volume: int):
        # Intern symbol strings to point to a single global string object in memory
        self.symbol = sys.intern(symbol)
        self.bid = bid
        self.ask = ask
        self.volume = volume

Results:

• Memory for 10,000,000 instances drops from 1.52 GB to 480 Megabytes (a 68.4% reduction).
• String interning eliminates 9,990,000 duplicate PyASCIIObject string allocations for repetitive ticker symbols like "AAPL", "MSFT", and "GOOG".
• Container memory stabilized safely under 700 MB.

5. Anti-Patterns & Common Pitfalls#

The `__del__` Destructor Memory Leak Anti-Pattern#

In Python versions prior to 3.4 (and even in modern versions under unmanaged cyclic references), defining a __del__ method on classes participating in cyclic references creates uncollectable garbage:

# HAZARDOUS ANTI-PATTERN
class LeakyResource:
    def __init__(self):
        self.circular_ref = None

    def __del__(self):
        # Destructor prevents the GC from guessing safe teardown order in cyclic graphs!
        print("Cleaning up resource...")

If two objects with __del__ methods reference each other, the garbage collector cannot determine which destructor to execute first. In older Python runtimes, both objects were moved to gc.garbage and leaked permanently in memory.

Remediation: Avoid __del__. Use explicit context managers (with statements) or weakref.finalize for deterministic resource reclamation.

Premature Optimization with C Extensions#

Rewriting algorithms in C or Cython before profiling often results in unmaintainable codebases that yield negligible performance gains if the underlying bottleneck was an I/O network delay or algorithmic inefficiency (e.g., an O(N^2) lookup that could have been resolved using a Python set with O(1) lookup).

6. Checkpoint Practice Problems#

The following four multi-part quantitative problems evaluate CPython small-object pool fragmentation mathematics, generational garbage collection tuning, __slots__ memory reduction calculations, and native C-extension GIL release scaling. Complete step-by-step solutions follow each problem prompt.

Problem 1: CPython Arena, Pool & Block Memory Allocation Mathematics#

Context:

A CPython 3.12 small object allocator (pymalloc) is servicing allocations for a high-frequency trading application. The application instantiates a massive batch of small custom telemetry struct objects.

Allocated Object Parameters:

• Object payload size: S = 58 bytes per object.
• Total objects instantiated: N = 250,000 objects.

`pymalloc` Architecture Parameters:

• Arena size: S_{arena} = 256 KB = 262,144 bytes.
• Pool size: S_{pool} = 4 KB = 4,096 bytes.
• Number of pools per arena: 4,096 bytes × 64 = 262,144 bytes (64 pools per arena).
• Pool header overhead: Each 4 KB pool reserves exactly H_{pool} = 48 bytes at the beginning of the pool for pool management metadata (pool_header). The remaining 4,096 - 48 = 4,048 bytes are divided into equal-sized blocks.
• Size classes: 64 classes, where Class k provides blocks of size:

Size(k) = (k + 1) × 8 bytes, for k ∈ \{0, 1, ..., 63\}.

Questions:

1. Identify the exact Size Class index k and block size assigned by pymalloc to an object of size S = 58 bytes.
2. Calculate the exact number of usable blocks contained within a single 4 KB pool for this size class.
3. Compute the internal fragmentation (wasted bytes per object) and the total unusable leftover slack bytes per pool.
4. Calculate the total number of 4 KB pools and 256 KB arenas required to store all N = 250,000 objects.
5. Compute the overall Memory Allocation Efficiency:

Efficiency = (Total Useful Payload Bytes / Total Allocated Arena Memory Bytes) × 100\%.

Solution:

Step 1: Size Class and Block Size Determination

Object size S = 58 bytes.

pymalloc rounds up to the nearest multiple of 8:

58 / 8 × 8 = 8 × 8 = 64 bytes.

Block size = 64 bytes.

Size class index k:

(k + 1) × 8 = 64 → k + 1 = 8 → k = 7 (Size Class 7).

Step 2: Number of Usable Blocks per Pool

Available pool memory after metadata header:

Available_pool = 4,096 - 48 = 4,048 bytes.

Number of 64-byte blocks per pool:

Blocks_per_pool = 4,048 / 64 = 63 blocks per pool.

Step 3: Internal Fragmentation and Leftover Slack per Pool

• Internal fragmentation per object = Block size - Requested payload size

Internal_frag = 64 - 58 = 6 bytes per object (9.375% per block).

• Pool leftover slack bytes = Available memory - (Blocks × Block size)

Slack_bytes = 4,048 - (63 × 64) = 4,048 - 4,032 = 16 bytes per pool.

Total non-payload bytes per pool = 48 (header) + 16 (slack) + (63 × 6) (fragmentation) = 48 + 16 + 378 = 442 bytes per pool.

Step 4: Total Pools and Arenas Required

Total objects N = 250,000.

• Number of pools required:

Pools_required = 250,000 / 63 = 3,968.25 = 3,969 pools.

• Number of arenas required:

Each arena houses 64 pools:

Arenas_required = 3,969 / 64 = 62.015 = 63 arenas.

Step 5: Overall Memory Allocation Efficiency

• Useful payload memory:

Useful_bytes = 250,000 × 58 bytes = 14,500,000 bytes (14.50 MB).

• Total allocated arena memory:

Allocated_bytes = 63 arenas × 262,144 bytes/arena = 16,515,072 bytes (16.515 MB).

• Memory Allocation Efficiency:

Efficiency = (14,500,000 / 16,515,072) × 100% ≈ 87.80%.

Conclusion: pymalloc delivers an outstanding 87.8% memory efficiency for small objects, eliminating system heap fragmentation while keeping internal waste below 12.2%.

Problem 2: Generational Garbage Collector Cycle Detection & Threshold Tuning#

Context:

A high-throughput API gateway running CPython 3.12 handles 1,000 HTTP requests per second. Each request creates temporary cyclic data structures (e.g., request context objects referencing authentication handlers that reference the request).

CPython GC Parameters:

• Standard default collection thresholds: gc.get_threshold() = (700, 10, 10).
• Generation 0 Threshold (T_0 = 700): A Gen 0 collection triggers when:

Δ_{alloc} = (Allocations - Deallocations) > 700.

• Generation 1 Threshold (T_1 = 10): A Gen 1 collection triggers when Generation 0 has been collected 10 times.
• Generation 2 Threshold (T_2 = 10): A Gen 2 collection triggers when Generation 1 has been collected 10 times (i.e., after 10 × 10 = 100 Gen 0 collections).

Workload Profiling:

• Each incoming HTTP request generates 5 net uncollected cyclic container objects.
• Collection execution latencies:

- Gen 0 collection: t_{gc0} = 0.2 ms.

- Gen 1 collection: t_{gc1} = 2.0 ms.

- Gen 2 collection (Full heap traversal): t_{gc2} = 45.0 ms. (Causes 45 ms latency spikes in P99 API responses!).

Questions:

1. At 1,000 requests per second, calculate how many Gen 0, Gen 1, and Gen 2 collections occur per minute under default thresholds.
2. Calculate the total CPU time consumed by garbage collection pauses per minute, and compute the percentage of time the application is paused by full Gen 2 collections.
3. The platform team tunes the thresholds to (5000, 20, 25). Calculate the new frequency of Gen 0, Gen 1, and Gen 2 collections per minute.
4. Calculate the new total pause time per minute under the tuned configuration and evaluate the impact on API latency spikes.

Solution:

Step 1: Collection Frequencies Under Default Thresholds `(700, 10, 10)`

• Requests per second = 1,000.
• Per minute: Requests = 1,000 × 60 = 60,000 requests/min.
• Net cyclic allocations per request = 5.
• Net cyclic allocation rate = 60,000 × 5 = 300,000 net container allocations per minute.

• Generation 0 Collections:

Gen 0 triggers every 700 net allocations:

Gen0_per_min = 300,000 / 700 ≈ 428.57 collections/minute (approx. 7.14 per second).

• Generation 1 Collections:

Triggers every 10 Gen 0 collections:

Gen1_per_min = 428.57 / 10 ≈ 42.86 collections/minute.

• Generation 2 Collections:

Triggers every 10 Gen 1 collections:

Gen2_per_min = 42.86 / 10 ≈ 4.29 collections/minute.

Step 2: Total Pause Time per Minute (Default)

• Gen 0 pause time: 428.57 × 0.0002 s = 0.0857 s (85.7 ms).
• Gen 1 pause time: 42.86 × 0.0020 s = 0.0857 s (85.7 ms).
• Gen 2 pause time: 4.29 × 0.0450 s = 0.1930 s (193.0 ms).

Total pause time per minute = 0.0857 + 0.0857 + 0.1930 = 0.3644 seconds/min (364.4 ms).

Latency Spike Impact: Every ~14 seconds, a full Gen 2 collection freezes the server for 45 ms, directly causing P99 SLA breaches.

Step 3: Collection Frequencies Under Tuned Thresholds `(5000, 20, 25)`

• Generation 0:

Gen0_per_min = 300,000 / 5,000 = 60.00 collections/minute (exactly 1 per second).

• Generation 1:

Triggers every 20 Gen 0 collections:

Gen1_per_min = 60.00 / 20 = 3.00 collections/minute.

• Generation 2:

Triggers every 25 Gen 1 collections:

Gen2_per_min = 3.00 / 25 = 0.12 collections/minute (approx. once every 8.33 minutes).

Step 4: Total Pause Time Under Tuned Configuration

• Gen 0 pause time: 60.00 × 0.0003 s (larger Gen 0 slightly increases sweep time) = 0.018 s.
• Gen 1 pause time: 3.00 × 0.0025 s = 0.0075 s.
• Gen 2 pause time: 0.12 × 0.0450 s = 0.0054 s.

Total pause time = 0.018 + 0.0075 + 0.0054 = 0.0309 seconds/min (30.9 ms).

Conclusion: Tuning GC thresholds reduced total pause time by 91.5% (from 364 ms down to 31 ms) and reduced full Gen 2 freeze frequency from once every 14 seconds to once every 8.3 minutes, stabilizing P99 latencies.

Problem 3: Memory Footprint Optimization with `__slots__`#

Context:

A spatial GIS application caches N = 4,000,000 geographic coordinates in memory on a 64-bit Linux server running Python 3.12.

Each coordinate object contains three attributes:

• latitude: float (64-bit IEEE 754 float)
• longitude: float
• elevation: float

Memory Layout Specifications on 64-bit CPython:

• PyObject base header: 16 bytes (ob_refcnt: 8 bytes, ob_type: 8 bytes).
• Standard Class Instance with __dict__:

- Instance struct: Base header (16 bytes) + __weakref__ pointer (8 bytes) + __dict__ pointer (8 bytes) = 32 bytes.

- PyDictObject container overhead: 152 bytes minimum.

- Total per standard instance = 32 + 152 = 184 bytes (excluding referenced float objects).

• Slotted Class Instance (__slots__ = ('latitude', 'longitude', 'elevation')):

- Instance struct: Base header (16 bytes) + three C-level PyObject* member pointers (3 × 8 = 24 bytes) = 40 bytes.

- Zero __dict__ overhead. Zero __weakref__ overhead. Total per slotted instance = 40 bytes.

• Each PyFloatObject: 24 bytes (16 bytes header + 8 bytes raw double).

Questions:

1. Calculate the total memory consumed by 4,000,000 coordinate instances using the standard class representation (including the float objects).
2. Calculate the total memory consumed using the __slots__ optimized representation.
3. Compute the absolute memory saved (in Megabytes and Gigabytes) and the percentage memory reduction.
4. If the server is capped at 1.0 GB of RAM, determine whether the standard representation fits in memory, and how much headroom the slotted version provides.

Solution:

Step 1: Memory Consumed by Standard Class Representation

For 1 standard instance:

• Instance struct + __dict__ overhead = 184 bytes.
• 3 distinct float objects: 3 × 24 bytes = 72 bytes.
• Total memory per coordinate = 184 + 72 = 256 bytes per coordinate.

For 4,000,000 coordinates:

Total_standard = 4,000,000 × 256 bytes = 1,024,000,000 bytes.

In Megabytes = 1,024,000,000 / (1024²) ≈ 976.56 MB (approx. 1.024 GB).

Step 2: Memory Consumed by `__slots__` Representation

For 1 slotted instance:

• Slotted instance struct = 40 bytes.
• 3 distinct float objects: 3 × 24 bytes = 72 bytes.
• Total memory per coordinate = 40 + 72 = 112 bytes per coordinate.

For 4,000,000 coordinates:

Total_slotted = 4,000,000 × 112 bytes = 448,000,000 bytes.

In Megabytes = 448,000,000 / (1024²) ≈ 427.25 MB (approx. 0.448 GB).

Step 3: Memory Reduction Calculations

• Absolute memory saved = 1,024,000,000 - 448,000,000 = 576,000,000 bytes (549.3 MB or 0.576 GB).
• Percentage reduction = (576,000,000 / 1,024,000,000) × 100% = 56.25% reduction.

Step 4: Server Capacity Assessment

• Available RAM: 1.0 GB (approx. 1,000,000,000 bytes).
• Standard representation: Requires 1.024 GB -> Exceeds 1.0 GB limit! Triggers OS swapping or immediate OOM kill.
• Slotted representation: Requires only 0.448 GB -> Fits easily, leaving 552 MB (55.2%) of free RAM headroom for application logic and caches.

Problem 4: C-Extension GIL Release & Multi-Core Speedup Analysis#

Context:

A cryptography microservice processes bulk batches of N = 1,000,000 cryptographic signature verifications on a server equipped with C = 8 physical CPU cores.

The engineer implements a C extension module fast_crypto. The batch verification routine takes T_{base} = 8.000 seconds of pure CPU computation in C.

Two compilation variants of the C extension are evaluated:

• Variant A (Locked): Does not release the GIL. The C code executes while holding the GIL throughout the entire 8 seconds.
• Variant B (GIL-Free): Wraps the core C calculation in Py_BEGIN_ALLOW_THREADS and Py_END_ALLOW_THREADS.

The workload is partitioned evenly across P concurrent Python worker threads (P ∈ \{1, 2, 4, 8\}).

• Overhead of spawning and synchronizing threads is negligible (τ ≈ 2 ms).
• Memory bus contention limits parallel efficiency to 95% on 2 cores, 90% on 4 cores, and 85% on 8 cores.

Questions:

1. Calculate the elapsed execution duration and effective speedup for Variant A across P = 1, 2, 4, 8 threads.
2. Calculate the elapsed execution duration and effective speedup for Variant B across P = 1, 2, 4, 8 threads.
3. Formulate the Amdahl's Law speedup curve for Variant B.
4. Explain why Variant B can scale across all 8 CPU cores while Variant A is serialized.

Solution:

Step 1: Variant A Execution Duration and Speedup (GIL Held)

In Variant A, the C extension holds the Global Interpreter Lock:

• Even though P native OS threads are spawned across 8 CPU cores, only ONE thread can execute the C extension at any time.
• The 8.000 seconds of computation is strictly serialized.
• Adding threads introduces thread context switching without concurrency.
• Execution duration:

- P = 1: 8.000 s -> Speedup = 1.00x

- P = 2: 8.000 s + 0.050 s (contention) = 8.050 s -> Speedup = 0.99x

- P = 4: 8.000 s + 0.120 s = 8.120 s -> Speedup = 0.98x

- P = 8: 8.000 s + 0.250 s = 8.250 s -> Speedup = 0.97x

Step 2: Variant B Execution Duration and Speedup (GIL Released)

In Variant B, Py_BEGIN_ALLOW_THREADS releases the GIL. The 8 threads execute their native C loops simultaneously on 8 physical CPU cores.

Duration formula:

T(P) = (T_{base} / P) / Efficiency(P)

• P = 1:

T(1) = 8.000 / 1.0 = 8.000 seconds -> Speedup = 1.00x

• P = 2 (Efficiency = 95% = 0.95):

T(2) = (8.000 / 2) / 0.95 = 4.000 / 0.95 ≈ 4.211 seconds -> Speedup = 8.000 / 4.211 = 1.90x

• P = 4 (Efficiency = 90% = 0.90):

T(4) = (8.000 / 4) / 0.90 = 2.000 / 0.90 ≈ 2.222 seconds -> Speedup = 8.000 / 2.222 = 3.60x

• P = 8 (Efficiency = 85% = 0.85):

T(8) = (8.000 / 8) / 0.85 = 1.000 / 0.85 ≈ 1.176 seconds -> Speedup = 8.000 / 1.176 = 6.80x

Step 3: Amdahl's Law Speedup Formulation

Let p be the parallelizable fraction of the task. Because the verification routine is 100% pure C computation (p = 1.0):

Theoretical Speedup S(P) = P.

Accounting for memory bus and cache coherence efficiency factor η(P):

S_{effective}(P) = P · η(P)

For P = 8: S = 8 × 0.85 = 6.80x.

Step 4: Architectural Conclusion

• Variant A fails to parallelize because the C extension operates as a single critical section guarded by CPython's global lock.
• Variant B achieves a 6.8x speedup on 8 cores because releasing the GIL drops the CPython runtime lock, allowing the Linux kernel to schedule all 8 threads across all 8 hardware cores at 100% CPU utilization.