Groq API is one of the best alternatives to OpenAI API, offering super-fast AI inference with a generous free tier. This tutorial will discuss step-by-step how to integrate Groq API with Python, from setup to creating a simple but powerful AI application.

What is Groq API?

Groq API is a cloud AI service that uses Groq LPU (Language Processing Unit), a specialized hardware designed for ultra-fast AI inference. Compared to traditional GPUs, Groq LPU can generate tokens up to 10x faster. Groq provides a free API that you can use for various needs, including chatbots, summarization, and text analysis.

Required Tools or Libraries

Python 3.x
groq
requests
python-dotenv

Preparation

Create an API Key

We can obtain an API Key from Groq by visiting the following link Groq Console.
After that, create an account using your Google account and navigate to the API Key menu on the dashboard.
Copy and save the API Key!

Install Library or Dependencies

Run the following command on your terminal:

pip install groq python-dotenv requests

Configure Environment Variable

After installing the library, create a .env file in the root directory of the project and paste the code below.

GROQ_API_KEY=your_api_key

Note: replace your_api_key with the API Key we got from Groq Console.

Implementation

Load ENV

Create a config.py file, then paste the code below.

import os
from dotenv import load_dotenv

load_dotenv()

GROQ_API_KEY = os.getenv('GROQ_API_KEY')

if not GROQ_API_KEY:
    raise ValueError("GROQ_API_KEY not found in environment variables")

AVAILABLE_MODELS = {
    'openai': 'openai/gpt-oss-120b',
}

DEFAULT_MODEL = AVAILABLE_MODELS['openai']

The config.py file is responsible for retrieving the API Key from the .env file and handling errors if the GROQ_API_KEY variable is not found.

Test & Setup Connection

Before creating a Chat Bot, we need to test the connection with the API from Groq first. Create a file named test_groq.py.

from groq import Groq
from config import GROQ_API_KEY, DEFAULT_MODEL

def setup_groq():
    try:
        client = Groq(api_key=GROQ_API_KEY)
        
        print("🔄 Testing Groq API connection...")
        
        chat_completion = client.chat.completions.create(
            messages=[
                {
                    "role": "user",
                    "content": "Say hello in Japanese!"
                }
            ],
            model=DEFAULT_MODEL,
            max_tokens=50,
            temperature=0.7
        )
        
        # Extract response
        response = chat_completion.choices[0].message.content
        
        print("✅ Connection Successfully!")
        print(f"🤖 Response: {response}")
        print(f"📊 Model used: {DEFAULT_MODEL}")
        
        return True
        
    except Exception as e:
        print(f"❌ Error: {e}")
        return False

if __name__ == "__main__":
    success = setup_groq()
    
    if success:
        print("\n🎉 Setup successful! Ready for development.")
    else:
        print("\n🔧 Check your API key and internet connection.")

Explanation of several parameters and functions from the code above:

Groq(api_key=GROQ_API_KEY) initializes the client.
client.chat.completions.create() sends the configuration to the client.
messages Array containing conversation history.
model AI model to be used.
max_tokens Maximum output token limit.

After that, open the terminal and run the file with the command below.

python test_groq.py

If the connection is successful, it will display output like this in the terminal. Test Connection Groq API & Python - NganggurDev

Creating a Chatbot

Now we can create a Chat Bot with features that can save conversation history, provide answers, and even save conversations in JSON format files. Create a chatbot.py file and paste the code below.

from groq import Groq
from config import GROQ_API_KEY, DEFAULT_MODEL
import json
from datetime import datetime

class GroqChatbot:
    def __init__(self, model=DEFAULT_MODEL):
        self.client = Groq(api_key=GROQ_API_KEY)
        self.model = model
        self.conversation_history = []
        self.total_tokens_used = 0
        
        print(f"🤖 Chatbot initialized with model: {model}")
    
    def add_message(self, role, content):
        message = {
            "role": role,
            "content": content,
            "timestamp": datetime.now().isoformat()
        }
        self.conversation_history.append(message)

        if len(self.conversation_history) > 20:
            system_messages = [msg for msg in self.conversation_history if msg["role"] == "system"]
            recent_messages = [msg for msg in self.conversation_history if msg["role"] != "system"][-18:]
            self.conversation_history = system_messages + recent_messages
    
    def set_system_prompt(self, system_message):
        self.conversation_history = [msg for msg in self.conversation_history 
                                   if msg["role"] != "system"]
        
        self.conversation_history.insert(0, {
            "role": "system",
            "content": system_message,
            "timestamp": datetime.now().isoformat()
        })
    
    def get_response(self, user_message):
        try:
            self.add_message("user", user_message)
            
            api_messages = [
                {"role": msg["role"], "content": msg["content"]} 
                for msg in self.conversation_history
            ]
            
            chat_completion = self.client.chat.completions.create(
                messages=api_messages,
                model=self.model,
                max_tokens=200,      
                temperature=0.7,
                top_p=0.9,
                stream=False
            )
            
            ai_response = chat_completion.choices[0].message.content
            
            if hasattr(chat_completion, 'usage'):
                self.total_tokens_used += chat_completion.usage.total_tokens
            
            self.add_message("assistant", ai_response)
            
            return ai_response
            
        except Exception as e:
            error_message = f"Error: {str(e)}"
            print(f"❌ {error_message}")
            return error_message
    
    def save_conversation(self, filename=None):
        if not filename:
            timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
            filename = f"conversation_{timestamp}.json"
        
        try:
            with open(filename, 'w', encoding='utf-8') as f:
                json.dump({
                    "model": self.model,
                    "total_tokens": self.total_tokens_used,
                    "conversation": self.conversation_history
                }, f, indent=2, ensure_ascii=False)
            
            print(f"💾 Conversation saved to {filename}")
            
        except Exception as e:
            print(f"❌ Error saving conversation: {e}")
    
    def get_stats(self):
        return {
            "total_messages": len(self.conversation_history),
            "total_tokens_used": self.total_tokens_used,
            "model": self.model
        }

if __name__ == "__main__":
    bot = GroqChatbot()
    
    bot.set_system_prompt(
        "You are a helpful and friendly AI assistant."
        "Answer in natural and easy-to-understand."
        "If you don't know the answer, say so honestly."
    )

    print("🚀 Groq Chatbot is ready to use!")
    print("💬 Type your message (type 'quit' to exit, 'save' to save)")
    print("=" * 60)
    
    while True:
        user_input = input("\n👤 You: ").strip()
        
        if user_input.lower() == 'quit':
            print("👋 See you next time!")
            break
        elif user_input.lower() == 'save':
            bot.save_conversation()
            continue
        elif user_input.lower() == 'stats':
            stats = bot.get_stats()
            print(f"📊 Stats: {stats}")
            continue
        elif not user_input:
            continue
        
        print("🤖 Bot:", end=" ")
        response = bot.get_response(user_input)
        print(response)
        print("-" * 60)

Explanation of the main features of Chat Bot:

Memory Management: Chatbot stores conversation history with limits to save tokens.
System Prompt: Can set personality and specific instructions for AI.
Token Tracking: Monitors token usage for budget control.
Conversation Export: Saves conversations to a JSON file.
Error Handling: Robust error handling for production use.

After that, open the terminal and run the chatbot.py file with the command below.

python chatbot.py

Groq API Chatbot Demo with Python

Demo Chatbot Groq AI with Python - NganggurDev Save Command Stats Command

Conclusion

By following this tutorial, we have successfully created a super-fast AI Chatbot using the Groq API in Python.

Thank you.