Accelerating 3D CNNs on FPGAs Using Hardware-Aware Pruning

July 28, 2026 •

FPGA Hardware Acceleration Deep Learning Machine Learning

# Accelerating 3D CNNs on FPGAs Using Hardware-Aware Pruning

Three-dimensional Convolutional Neural Networks (3D CNNs) are widely used for spatial-temporal tasks such as video analysis and volumetric medical image processing. However, their compute and memory intensity pose significant challenges when deploying on edge and cloud FPGAs.

In our research at **Northeastern University**, we investigated hardware-aware pruning strategies designed specifically for FPGA acceleration targets.

## Key Challenges in 3D CNN Acceleration

1. **High Memory Bandwidth Demand**: 3D convolutions require processing volumetric kernel weights and feature maps, leading to frequent off-chip memory accesses (DRAM/HBM).
2. **Resource Constraints**: On-chip block RAM (BRAM/URAM) on current FPGAs cannot fit full 3D activation volumes without aggressive memory reuse or quantization/pruning.
3. **Irregular Pruning Overhead**: Traditional unstructured pruning creates sparse matrices that introduce control-flow overhead and degrade parallel execution on FPGA DSP blocks.

## Hardware-Aware Pruning Approach

To overcome these bottlenecks, we introduced a **hardware-aware structured pruning** methodology:

- **Block-Structured Sparsity**: We enforce fine-grained block sparsity that matches the SIMD vector width of our FPGA compute engines.
- **Resource Allocation Tuning**: Pruning ratios are determined dynamically based on available DSPs and BRAM tile capacities.
- **Roofline Model Optimization**: We balance compute efficiency (FLOPs per cycle) against memory access latency.

```
+-------------------+      +-----------------------+      +-------------------+
|  Original 3D CNN  | ---> | Hardware-Aware Pruner | ---> | Optimized FPGA    |
|  Model (Dense)    |      | (SIMD Vector Aligned) |      | Bitstream/Engine  |
+-------------------+      +-----------------------+      +-------------------+
```

## Performance Results & Impact

- **Compression Ratio**: Achieved up to 4.5x reduction in model parameter count with less than 0.5% accuracy drop.
- **Throughput Gain**: Realized significant frame-rate speedups compared to standard unpruned GPU/FPGA baselines.

For complete benchmarks and architectural details, check out our paper on [Google Scholar](https://scholar.google.com/citations?user=ZD-tEgMAAAAJ&hl=en).

Loading article...