Generative AI - Learn AI | TechLead

What is Generative AI?

Generative AI refers to AI systems that can create new content — text, images, code, audio, video — rather than just analyzing or classifying existing data. While traditional AI answers "what is this?", generative AI answers "what could this look like?"

🎨 The Generative Revolution:

From ChatGPT generating text to DALL·E creating images to GitHub Copilot writing code — generative AI has fundamentally changed how we create content and build software.

Types of Generative Models

📝 Large Language Models (LLMs)

Transformer-based models trained on massive text corpora to generate human-like text.

How they work:

Predict the next token in a sequence. Trained with self-supervised learning on trillions of tokens. Fine-tuned with RLHF for helpful responses.

Examples:

GPT-4, Claude, Gemini, Llama, Mistral. Applications: chatbots, code generation, summarization, translation.

🖼️ Diffusion Models

Generate images by learning to reverse a noise-adding process.

The Process:

1. Forward: Gradually add Gaussian noise to an image until it becomes pure noise
2. Reverse: Train a neural network to predict and remove the noise step by step
3. Generate: Start from pure noise and iteratively denoise to create new images

Examples: Stable Diffusion, DALL·E, Midjourney, Imagen

🎭 GANs (Generative Adversarial Networks)

Two networks compete: a generator creates fakes, a discriminator detects them.

How it works:

The generator tries to create realistic data. The discriminator tries to tell real from fake. They train adversarially until the generator produces data indistinguishable from real.

Applications: StyleGAN (face generation), image super-resolution, style transfer

🔄 VAEs (Variational Autoencoders)

Learn a compressed (latent) representation of data and generate new samples from it.

Architecture:

Encoder: maps input to a probability distribution in latent space. Decoder: samples from latent space and reconstructs output. The latent space is continuous and smooth, enabling interpolation.

Using Generative AI APIs

Modern generative AI is accessible through APIs. Here's how to use them in JavaScript:

// Using OpenAI API for text generation
const response = await fetch('https://api.openai.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
  },
  body: JSON.stringify({
    model: 'gpt-4',
    messages: [
      { role: 'system', content: 'You are a helpful coding assistant.' },
      { role: 'user', content: 'Explain React hooks in 3 sentences.' }
    ],
    temperature: 0.7,  // Controls creativity (0=deterministic, 2=very random)
    max_tokens: 200,
  }),
});

const data = await response.json();
console.log(data.choices[0].message.content);

// Streaming responses for better UX
const stream = await fetch('https://api.openai.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'Content-Type': 'application/json',
    'Authorization': `Bearer ${process.env.OPENAI_API_KEY}`,
  },
  body: JSON.stringify({
    model: 'gpt-4',
    messages: [{ role: 'user', content: 'Write a short poem about coding.' }],
    stream: true,
  }),
});

const reader = stream.body.getReader();
const decoder = new TextDecoder();

while (true) {
  const { done, value } = await reader.read();
  if (done) break;

  const chunk = decoder.decode(value);
  const lines = chunk.split('\n').filter(line => line.startsWith('data: '));

  for (const line of lines) {
    const data = line.replace('data: ', '');
    if (data === '[DONE]') break;
    const parsed = JSON.parse(data);
    process.stdout.write(parsed.choices[0]?.delta?.content || '');
  }
}

Key Generation Parameters

🌡️ Temperature

Controls randomness. 0: deterministic (always picks most likely token). 1+: more creative/random. Use 0 for factual tasks, 0.7-1.0 for creative writing.

🎯 Top-P (Nucleus Sampling)

Only consider tokens whose cumulative probability reaches P. top_p=0.9: considers the smallest set of tokens that sum to 90% probability.

📏 Max Tokens

Maximum number of tokens to generate. Controls output length and cost. One token ≈ 0.75 English words.

🔄 Frequency Penalty

Penalizes repeated tokens to encourage diverse outputs. Range: -2.0 to 2.0. Higher values reduce repetition.

Generative AI for Code

How AI Code Generation Works

• Training: LLMs trained on billions of lines of open-source code from GitHub
• Context: Uses surrounding code, comments, function signatures as prompts
• Completion: Predicts the most likely next tokens to complete your code
• Tools: GitHub Copilot, Cursor, Codeium, Amazon CodeWhisperer

Best practices for AI-assisted coding:

✓ Write clear comments describing what you want before the code
✓ Review every suggestion — AI can hallucinate APIs that don't exist
✓ Use it for boilerplate, tests, and documentation more than core logic
✓ Understand the code it generates — don't blindly accept suggestions

🔑 Key Takeaways

• LLMs generate text by predicting the next token; diffusion models generate images by denoising
• Temperature and top-p control the creativity-accuracy tradeoff
• GANs use adversarial training; VAEs use latent space encoding
• Streaming API responses create much better user experiences
• AI code tools are most effective when you write clear intent in comments