How Homomorphic Encryption Now Runs CIFAR-10 Inference in 200ms

Fully homomorphic encryption lets a server compute on ciphertext directly. You send an encrypted CIFAR-10 image, the server runs a ResNet-20 over it, and you get back an encrypted label. The server never decrypts anything and never learns the picture or the answer.

The catch has always been speed. An early published baseline needed 180.4 seconds on a 64-thread CPU to classify a single 32x32 image, and older reference implementations ran into the thousands of seconds. That is the number the whole field has spent years attacking.

Getting from three minutes per image down to something interactive is a story of three concrete mechanisms, and then a startup that claims to have leapfrogged all of them.

CKKS, the scheme most of this work uses, handles addition and multiplication on encrypted vectors natively. That covers convolutions and matrix multiplies fine. The problem is the nonlinear parts of a network, above all ReLU.

A comparison like max(x, 0) has no direct FHE equivalent, so it gets approximated by a high-degree polynomial. Every multiplication in that polynomial eats multiplicative depth, and when the depth runs out you must bootstrap: an expensive refresh that resets the ciphertext's noise budget.

In the SNU group's analysis of prior work, bootstrapping plus ReLU accounted for 84% of total execution time. Attack that 84% and everything else is rounding error, which is exactly what the next mechanisms do.

Under the hood, CKKS operations reduce to polynomial arithmetic over huge rings, and the hot loop is the Number Theoretic Transform (NTT) plus base conversion. These are massively parallel, which makes them a natural fit for a GPU.

Porting the CPU implementation to an A100 gave the SNU group a 21.0x speedup on its own, before any algorithmic cleverness. Standalone kernel benchmarks tell the same story: one RTX 4090 library reported up to 246x on homomorphic multiply and 161x on relinearization versus CPU.

Raw hardware gets you a factor of 20-ish. The rest of the journey to one second is about doing less work in the first place.

If ReLU is what forces the expensive high-degree approximation and the bootstraps, the direct fix is to stop using ReLU. SNU's AESPA replaces the ReLU-plus-batchnorm block with a low-degree Hermite polynomial activation, a quadratic, and trains the network end-to-end with it.

The worry is always accuracy. Here it barely moved: ResNet-20 on CIFAR-10 hit 92.18% with AESPA against 92.15% for the ReLU baseline. A cheaper activation, essentially free.

A quadratic consumes far less multiplicative depth than a degree-30 polynomial, so the network needs fewer bootstraps to get through all its layers. That is where the largest chunk of that 84% starts to disappear.

A CKKS ciphertext holds thousands of slots, so a whole feature map is packed into one encrypted vector. To line up values for a convolution you rotate the slots, and rotation is not cheap. In the referenced Lee et al. baseline, rotations ate more than 83% of convolution time.

A single fixed packing layout either minimizes bootstraps or minimizes rotations, rarely both. HyPHEN uses multiple packing methods and switches the layout before and after each convolution, picking whichever arrangement is cheapest for the next operation.

AESPA and HyPHEN together added a 6.12x speedup on top of the GPU port. Stack the two factors and the arithmetic lands almost exactly where the paper says it does.

Combined, GPU kernels plus AESPA plus HyPHEN bring ResNet-20 CIFAR-10 FHE inference to 1.4 seconds on an A100. Measured against the old reference implementation, that is a 1,622x improvement, roughly 21 x 6.12 from the two factors above with the rest from tuning.

The same stack, simulated on a proposed custom accelerator called SHARP, projects to 0.03 seconds. That 30ms figure is the closest published number to "interactive," and it is a hardware simulation, not a measured GPU run. The technique also scales up: ResNet-18 on ImageNet lands at 14.7 seconds on GPU.

So the honest, peer-reviewed frontier for a measured GPU is about 1.4 seconds. Which is what makes the next claim so eye-catching.

Belfort Labs, a startup from KU Leuven-affiliated researchers, announced sub-200ms CIFAR-10 FHE inference via a blog post and live demo, powered by an upcoming GPU library they call Cyclops. If real and comparable, that is roughly 7x faster than the published 1.4s A100 record. No paper, no methodology, and no architecture details have been released.

The demo also sidesteps the hardest production cost. On Hacker News, co-founder "furkanturan" confirmed it pre-caches the same evaluation key material for every visitor instead of generating per-user keys, to avoid a slow setup on mobile. So the number reflects raw compute, not an end-to-end trust flow.

Because the client's secret is downloaded from Belfort's own server, an outsider cannot cryptographically prove the box is doing FHE at all rather than decrypting and computing in plaintext. That is not evidence of fraud, but it is not verifiable either. Commenters also clocked the ciphertext ballooning about 341x and saw real misclassifications, both consistent with known FHE overhead and ResNet-20's ~90% accuracy ceiling.

And key material is the unsolved part. As FHE researcher j2kun noted in the thread, each user needs at least ~100 MiB of evaluation keys; a billion-user service at 1 GiB each is an exabyte of storage, a problem the demo simply does not touch.