A CUDA Graph inference engine for generative recommendation supporting wide beam search and FP8 serving, solving recommendation latency and throughput bottlenecks. 85 stars show AI infrastructure interest; a recommendation service or open standard could commercialize, but NVIDIA Merlin and in-house systems compete, and ecosystem lock-in is the risk.
FlashRec is a CUDA-graph engine for generative recommendation: wide beam search (3–5 SID steps, n=50–512+) over a trie-constrained catalog, in-process FP8 serving, and ranked beams on /v1/chat/complet…
★85