An RTX 5090 on a Mac Mini Runs at 7.48 Tokens Per Second. That's the Breakthrough.

Seven years. That's how long Mac users who wanted CUDA had to wait. Apple dropped NVIDIA support in macOS Mojave back in 2018, and the answer to "can I run serious AI on a Mac?" was always the same: buy a PC, rent a cloud box, or settle for Metal. Then, at the end of March 2026, Apple approved TinyGPU. Not a hack. Not a jailbreak. An open-source DriverKit extension from George Hotz's Tiny Corp that lets M-series Macs drive NVIDIA Ampere-and-newer and AMD RDNA3-and-newer GPUs over Thunderbolt and USB4, signed off by Apple with no SIP disabling required. The Hacker News thread barely registered, but in the AI hardware world it was an earthquake. Approved, however, is not the same as fast. We combed through the benchmark reports, the GitHub issue tracker, the community threads, and the manufacturer statements to figure out what this stack actually delivers in mid-2026 — and whether the RTX 5060 family, the budget Blackwell cards most cost-conscious AI tinkerers are eyeing, is the right companion for it.

First, untangle the name. Two distinct projects share the "TinyGPU" label. One is a 240,000-transistor ASIC that renders 3D graphics at up to 15 FPS — the world's "tiniest GPU," a delightful piece of open-source hardware engineering, and entirely unrelated to AI inference. The one everyone is actually talking about is Tiny Corp's macOS driver extension, built on top of the tinygrad neural network framework. This review is about the driver. The driver is best understood as a clean-sheet rewrite of GPU support for macOS. The GPU appears as a raw compute device through its memory-mapped PCIe interface. That design choice carries a hard consequence: TinyGPU is strictly for AI and ML workloads. No gaming, no external displays, no video editing acceleration. As one TechEnclave user put it: "It won't help you with gaming, video editing, or even driving an external monitor — the Mac doesn't see it as a traditional graphics card." A few architectural details shape everything that follows. The driver runs through official system extension mechanisms, which is why Apple signed off. NVIDIA cards need Docker Desktop to run the NVCC compiler — an extra setup step and, at least in theory, an extra performance tax. And because it's compute-only by design, the scope stays manageable but rules out a whole class of uses. Installation, at least, is genuinely painless. YouTuber Alex Ziskind, whose hands-on testing has become the de facto reference for this project, reported getting cards running on a Mac Mini M4 Pro in "less than five minutes." That's the easy part.

The RTX 5060 family: what Blackwell does at the low end

The hardware side of this story matters because the RTX 5060 family is where most people actually land. It's the entry point of NVIDIA's consumer Blackwell generation, and it brings fifth-generation Tensor Cores that meaningfully improve INT8 throughput. | GPU | VRAM | Memory bandwidth | TDP | Est. price | |---|---|---|---|---| | RTX 5060 | 8GB GDDR7 | ~384 GB/s | 130W | ~$249-349 | | RTX 5060 Ti 16GB | 16GB GDDR7 | ~448 GB/s | ~150W | ~$429-570 | There's a quirk in the reported FP32 numbers for this generation: test runs show the 5060 Ti at 22.7 TFlops while the 5070 Ti lands around 342 TFlops — a 15x jump. It reflects architectural differences in how these numbers are measured rather than a typo: the midrange Blackwell parts expose full FP32 throughput in ways the entry-level cards don't. Chinese tech outlet ZOL measured Llama-3-8B INT8 inference at 142 tokens per second on the 5060, a 31% improvement over the previous generation at the same price point. Community runs of Llama 3.1 8B at Q4_K_M land closer to 30 tokens per second. That gap is a reminder that quantization, context length, and measurement methodology matter at least as much as the silicon. Either way, the 5060 is a legitimate step up for local inference. MoE models change the value equation further. A Hugging Face run of Ourbox-35B — a 35B-class architecture with only 3B active parameters — hit 20.01 tokens per second on the 8GB laptop version of the 5060. As one Hacker News user noted: "The whole point of an A3B model is that decode cost scales with active parameters (~3B), not total (34.7B)." In practice, the experts sit in system RAM while the attention, router, and shared layers occupy VRAM. An 8GB card can run a model that would seem to need 35GB. That trick alone makes the 5060 more interesting than its VRAM suggests. Fine-tuning is viable too. ResNet-50 single-card training throughput improved 37% with FP16 mixed precision and CUDA 12.4 support, and several GitHub projects report successful LoRA/QLoRA runs on the 8GB card for lightweight security-domain LLMs. Stable Diffusion LoRA training has been observed running eight hours without throttling, with thermals steady under 72°C. The 16GB 5060 Ti is the more interesting card for LLM work. Hardware Corner calls it "one of the best LLM values" at $429, and a dual-card setup — 32GB of VRAM for roughly $858 — can "comfortably handle 30B-32B parameter models with longer context windows than the RTX 3090 can sustain," without the 3090's appetite for power, heat, and case space. The 5060 Ti runs off a single 8-pin connector and a 550W power supply, fits into 229mm, and doesn't involve gambling on a used card. For a generation of builders burned by the secondhand 3090 market, that's a real selling point.

The TinyGPU reality check

Now the part that bruises enthusiasm. Ziskind's testing connected RTX 5060 Ti, 5070 Ti, and 5090 cards to a Mac Mini M4 Pro through Thunderbolt/USB4 eGPU enclosures. The headline result: the RTX 5090, running Llama 3.1 8B through TinyGPU, delivered roughly 7.48 tokens per second. That same card on a native Windows or Linux system would deliver throughput in a completely different league. The gap suggests the current driver stack is nowhere near extracting the card's native potential. But there's a nuance buried in the numbers: time-to-first-token came in three to four times faster than native Metal inference. The model starts talking almost immediately; sustained throughput just never catches up. Ziskind's FP32 matrix multiplication runs told a similar story. The 5060 Ti measured 22.7 TFlops, while the 5070 Ti and 5090 both came in around 342 TFlops. Those figures are what the harness reported through the driver, not official spec-sheet numbers. The point is that the compute pipeline is genuinely functional — but it's not extracting anything close to what the silicon can deliver. A TechSpot commenter diagnosed the bottleneck precisely: "The real culprit is kernel efficiency: the 5090's memory can do 1.8TB/s, but the driver is only reaching 33GB/s." Another reply provided the necessary context: "This isn't meant to compete with llama.cpp right now. The point is that the driver, compiler pipeline, and memory management are all in place." Plugable, an eGPU enclosure maker that tested TinyGPU with an AMD Radeon 9070 and an RTX 5070 in its Thunderbolt 5 enclosure, came back blunter: "We didn't see much improvement over the baseline system performance." One thing worth stressing: Thunderbolt bandwidth isn't the limiter. Thunderbolt 5's 80Gbps (up to 120Gbps in some implementations) has drastically narrowed the gap between external and internal GPUs — 3DMark Time Spy scores within about 6% of desktop cards — but that's for gaming-class workloads on mature drivers. For TinyGPU, the constraint is kernel generation quality, not cable bandwidth.

The issue tracker is the honest spec sheet

The project's true status lives in its GitHub issues, and two entries stand out. Issue #14338: an RTX 5060 Ti (GB206/Blackwell) fails to initialize over USB4/Thunderbolt on a MacBook Pro M1. The GPU's GSP (GPU System Processor) fails during Display Engine initialization with FBFLCN errors. The working theory is that the 570.144 firmware lacks USB4/TB4 bus type support — constants like NV2080_CTRL_INTERNAL_EGPU_BUS_TYPE_USB4 only exist in 580.x firmware builds. The issue remains open after multiple configuration attempts. Issue #15813: on a Mac Mini M4 with a Sonnet Breakaway Box, macOS detects the GPU on the PCIe bus but never assigns BAR addresses, hanging the TinyGPU server at the probe stage. As the reporter explained: "Because BARs are unassigned, any MMIO read returns 0xFFFFFFFF (PCIe Master Abort)." The device is visible but has no memory apertures to talk to. "We never get past probe because the device has no memory apertures at all." These are what early-stage drivers look like in public. On the positive side, the project has confirmed working: driver extension activation, socket communication, BAR access in some configurations, VRAM tests, and GSP FMC/COT boot. Tiny Corp has also shipped an eGPU adapter board based on ADT-Link hardware — the thing you'd actually buy to connect a card to a Mac — and keeps iterating.

What you can actually do with this today

Despite the rough edges, several real workloads are viable. Multi-card configurations are further along than you'd expect. A 4x RTX 5060 Ti tensor-parallel setup ran Qwen3.6-27B with 262,144-token context windows, no KV cache quantization: | Context length | Decode (tok/s) | With MTP (tok/s) | Speedup | |---|---|---|---| | 1K | 39.4 | 69.0 | 1.75x | | 64K | 35.3 | 81.0 | 2.29x | | 255K | 26.7 | 52.2 | 1.96x | The developer's bottleneck note is instructive: "collective latency, not the interconnect." The cards lack peer-to-peer transfers, and yet PCIe utilization only reached 15-20% — the GPUs spend most of their time stalled on per-layer all-reduce synchronization. Speculative decoding is a mixed bag. On the 5060 Ti, it delivered a 1.85x speedup on 14B math models. But on a fast 7B model, the CUDA draft collapsed to 0.27x of baseline. A GitHub commenter noted that the Vulkan build carries much lower speculative overhead and "stayed >= baseline where CUDA cratered" — so on fast small models, pick the Vulkan backend or skip drafting entirely. There's an efficiency angle too. The arXiv evaluation of Blackwell consumer GPUs for small-business LLM deployment (2601.09527) found NVFP4 quantization delivers 1.6x throughput over BF16 with 41% lower energy consumption, at a 2-4% quality cost. For a 130W card that can already sit in a NUC-class workstation, that's a meaningful combination. What you can't do yet matters equally. Full 70B models on a single 5060? Only with heavy quantization. Production-grade throughput through TinyGPU on macOS? Not until kernel efficiency improves. Anything involving a display? Compute only, remember. And if you're on a MacBook Pro M1, don't expect 50-series support until the firmware issues shake out.

The economics of a budget Blackwell AI box

The same academic study mapped the cost case for self-hosted inference: $0.001-0.04 per million tokens in electricity, which is 40-200x cheaper than budget-tier cloud APIs. At moderate volume — 30 million tokens per day — hardware pays for itself in under four months. The RTX 5090 delivers 3.5-4.6x higher throughput than the 5060 Ti with 21x lower latency for RAG workloads, but budget GPUs win throughput-per-dollar for API-style workloads at sub-second latency. The 5060 family's argument isn't raw speed; it's cost per token. One caveat: 2026 has been rough for GPU buyers. GDDR memory shortages have pushed prices up across the RTX 50 series. In China, the RTX 5060 surged 500-800 yuan in four days in August, with street prices landing at 2,700-3,100 yuan. Used 5060 Ti and 5070 Ti prices rose 370 and 600 yuan respectively in July. The "budget AI card" is a moving target, and memory supply — not compute demand — is setting the price.

Community temperature

The Bilibili tech community captured the historical weight of TinyGPU in one line: "What two trillion-dollar companies wouldn't do, an open-source community accomplished." That's the significance, and it's worth sitting with. But the day-to-day conversation is more measured. The Hacker News thread around the announcement was small — four points and minimal discussion. The real work happens on GitHub, where the focus stays on compatibility and kernel quality. The gap between the breakthrough and the experience is wide, and the people closest to the project acknowledge it openly.

The verdict

TinyGPU is a foundation, not a finished product. What's striking is how complete the foundation is: the driver, the compiler path, the memory management, the hardware link. What's missing is optimization, which is an engineering problem rather than a feasibility question. For the RTX 5060 family, the answer stands independently: these are the most capable entry-level AI GPUs available, and the 5060 Ti 16GB in particular is one of the best LLM values on the market. TinyGPU just isn't the reason to buy one — not yet. The question isn't whether TinyGPU plus an RTX 5060 can work. It clearly can, in the right configurations. The question is whether you want to be an early adopter on a project that's still in its "make it work, then make it fast" phase. If you're patient, the hardware you buy today will only get faster as the driver matures. If you need production-grade throughput this year, buy the card, skip the Mac, and check back in on TinyGPU in twelve months.