Groq

About Groq

Groq was founded in 2016 by former Google engineers who built the original Google TPU, and has developed a fundamentally different approach to AI chip design: the Language Processing Unit (LPU). Unlike GPUs, which are parallelised matrix computation units repurposed for AI, LPUs are designed specifically for the sequential token-by-token generation process that underlies all large language model inference. The result is dramatically lower latency and higher throughput for inference workloads compared to GPU-based alternatives. Groq's GroqCloud API makes this hardware advantage available to developers via a simple REST API. The service hosts open-source models including Llama 3 (8B, 70B, and 405B), Mixtral 8x7B and 8x22B, Gemma, and others, with token generation speeds that commonly exceed 500 tokens per second for large models—compared to 30-80 tokens per second for typical GPU inference. This speed advantage is meaningful for use cases where real-time interaction, streaming, or high-volume batch processing are requirements. For developers building AI applications, Groq offers an OpenAI-compatible API, making migration straightforward. Pricing is per-token and competitive with alternative open-source inference providers, with a generous free tier for experimentation. The company has attracted significant enterprise interest from developers optimising latency-sensitive applications such as voice AI, real-time coding assistants, and customer-facing agents. However, for European businesses with GDPR compliance requirements, Groq presents meaningful concerns. Groq Inc. is incorporated in California, United States, and subject to the CLOUD Act. Current data centre infrastructure is located in the United States. While Groq has published a privacy policy and GDPR compliance documentation, there are no EU data residency options as of early 2026, no published ISO 27001 or SOC 2 Type II certifications, and inference of personal data through Groq's API requires appropriate GDPR transfer mechanisms (SCCs). European organisations should evaluate Groq for non-personal-data inference use cases, or for development and prototyping contexts where GDPR transfer constraints are managed appropriately.