01
Forms of Parallelism
- Instruction-level parallelism overlaps processor instructions.
- Data parallelism applies the same operation across many data elements.
- Task parallelism executes different tasks concurrently.
- Thread-level parallelism distributes work among execution threads.
- Distributed parallelism coordinates work across networked machines.
02
Amdahl's Law
speedup ≤ 1 / (s + (1-s)/N)
If fraction s of a workload is inherently serial, increasing the number of parallel workers N has a bounded benefit.