Next.js to TanStack Start & Convex Migration for AI Thumbnail Generation

Introduction

This documentation outlines the migration of Subfast, an AI YouTube thumbnail generation tool, from a Next.js and Prisma stack to a new architecture utilizing TanStack Start and Convex. This shift aims to deliver superior performance, reduce operational costs, and significantly enhance the developer experience through real-time data synchronization and simplified backend management.

Configuration Checklist

Element	Version / Link
Language / Runtime	TypeScript, Node.js
Web Framework (Old)	Next.js
Web Framework (New)	TanStack Start
Database (Old)	PostgreSQL (via Prisma)
Database (New)	Convex
Job Queue (Old)	Qstash
Job Queue (New)	Convex
Authentication	BetterAuth
Mobile Build	Expo
Styling	Tailwind CSS
Payments	Stripe
AI APIs	OpenAI API Key, Google Gemini API Key

Step-by-Step Guide

Step 1 — Accessing the Dashboard and Navigating

The new stack, powered by TanStack Start and Convex, offers instant loading of pages and real-time data synchronization, providing a smoother user experience compared to the previous Next.js stack.

// No specific code command for this step, as it's UI interaction.
// The user navigates to beta.thumbfa.st and logs in.
// The dashboard loads instantly due to server-side rendering and Convex's real-time capabilities.

Step 2 — Creating a New Thumbnail with Instant UI Updates

When creating a new thumbnail, the interface for adding images, style references, and persons loads instantaneously, showcasing the client-side responsiveness of TanStack Start combined with Convex's efficient data handling.

// No specific code command for this step, as it's UI interaction.
// The user clicks "Create Thumbnail" or "New Thumbnail".
// The input fields and options for image, style, and person references appear immediately.

Step 3 — Generating Thumbnails with Real-time Progress

The generation process leverages Convex's backend functions and real-time database. Unlike the old stack which required polling, Convex pushes updates directly to the client, allowing for instant display of generation progress and final images as they become available.

// This is a conceptual representation of the interaction with Convex.
// The actual generation logic resides in Convex backend functions.

// On the client-side (TanStack Start):
// When the user clicks 'Generate', a Convex mutation is called.
const generateThumbnail = async (prompt, images, style, person) => {
  // Call a Convex backend function (mutation) to start the generation job.
  // Convex will handle the background job and update the database.
  await convex.mutation("thumbnails:generate", { prompt, images, style, person });
};

// On the server-side (Convex backend function - e.g., convex/thumbnails.ts):
// This function is triggered by the client and runs in the background.
export const generate = mutation({
  args: {
    prompt: v.string(),
    images: v.array(v.string()),
    style: v.string(),
    person: v.string(),
  },
  handler: async (ctx, args) => {
    // 1. Create a new thumbnail entry in the database with a 'pending' status.
    const newThumbnailId = await ctx.db.insert("thumbnails", {
      status: "pending",
      // ... other thumbnail data
    });

    // 2. Trigger an AI image generation process (e.g., via OpenAI/Gemini API).
    const generatedImages = await callAIGenerationService(args); // [Editor's note: callAIGenerationService would be an external API call to an AI model]

    // 3. Update the thumbnail entry in the database with the generated images and 'completed' status.
    // Convex automatically synchronizes this update to all subscribed clients in real-time.
    await ctx.db.patch(newThumbnailId, {
      status: "completed",
      images: generatedImages,
    });

    return newThumbnailId;
  },
});

// On the client-side (TanStack Start):
// The UI subscribes to changes in the 'thumbnails' table.
// When the 'status' changes from 'pending' to 'completed', the new image is displayed instantly.
const { data: thumbnails } = convex.useQuery("thumbnails:list");
// The UI component would re-render automatically when 'thumbnails' data changes.

Comparison Tables: Next.js vs. TanStack Start & Convex

Feature	Old Stack (Next.js + Prisma/PostgreSQL + Qstash)	New Stack (TanStack Start + Convex)
Data Fetching	Polling for updates, pre-fetching pages (can be costly)	Real-time synchronization via Convex, no polling needed
Cost	Higher due to extensive pre-fetching and manual infrastructure management	Lower, as data is fetched on demand and Convex optimizes backend operations
Complexity	More complex to manage data synchronization (caching, invalidation), debugging logs, and separate job queues	Simplified backend (Convex handles database, jobs, automation), easier to maintain
Rendering	Mix of Client-Side Rendering (CSR) and Server-Side Rendering (SSR)	Primarily Server-Side Rendering (SSR) for faster initial loads and real-time updates
Data Consistency	Potential for unsynchronized data if caching/invalidation is not perfectly managed	Always synchronized across all clients due to Convex's real-time nature
Background Jobs	Requires external job queue systems like Qstash for long-running tasks	Convex natively handles background jobs and retries, simplifying workflow
Developer Experience	More manual state management and data handling	Streamlined, less boilerplate, automatic data synchronization

⚠️ Common Mistakes & Pitfalls

1. High pre-fetching costs in Next.js: Next.js's pre-fetching can lead to unnecessary data fetches, increasing operational costs, especially for dynamic content.
   • Fix: Migrate to a stack like TanStack Start + Convex where data is fetched on demand and synchronized in real-time, eliminating wasteful pre-fetching and reducing server load.
2. Data synchronization issues: Managing cache invalidation and ensuring data consistency across client and server in Next.js + Prisma can be complex and error-prone, leading to stale data in the UI.
   • Fix: Utilize Convex's real-time database and backend functions, which automatically synchronize data across all clients. This ensures immediate consistency without manual cache management or complex invalidation strategies.
3. Complex debugging and logging: Debugging issues in a traditional Next.js + PostgreSQL setup often involves sifting through distributed logs and manually tracking data flow across different services.
   • Fix: Leverage Convex's integrated backend and real-time capabilities, which provide a unified view of data and operations. This simplifies debugging, making it easier for developers and AI agents (like Coder) to identify and fix issues quickly.
4. Managing background jobs: Separating long-running tasks (like AI image generation) into external queue systems (e.g., Qstash) adds architectural complexity, increases latency, and introduces potential failure points.
   • Fix: Integrate background jobs directly within Convex functions. Convex handles retries and ensures job completion, simplifying the workflow and reducing external dependencies and the need for manual polling from the client.

Glossary

Pre-fetching: A technique where web pages or resources are downloaded in the background before the user explicitly requests them, aiming to improve perceived loading speed.
Polling: A method where a client repeatedly requests information from a server at regular intervals to check for updates.
Server-Side Rendering (SSR): A rendering method where the server generates the full HTML for a page on each request, sending a fully formed page to the client.
Client-Side Rendering (CSR): A rendering method where the browser downloads a minimal HTML shell and then fetches and renders the content using JavaScript.
Convex: A backend-as-a-service platform that provides a real-time database, serverless functions, and automatic data synchronization for web applications.
TanStack Start: A web framework (likely referring to TanStack Router or similar components used for building web applications) that emphasizes type safety and developer experience.

Key Takeaways

• Migrating to TanStack Start + Convex significantly improves application performance and responsiveness due to real-time data synchronization.
• Convex simplifies backend development by integrating database, serverless functions, and real-time updates into a single, cohesive platform.
• The new stack reduces infrastructure costs by eliminating the need for constant data polling and wasteful pre-fetching.
• Developer experience is greatly enhanced through a more streamlined architecture, automated data management, and simplified debugging.
• Convex's integrated job system handles long-running tasks and retries automatically, improving application reliability and reducing manual oversight.
• The adoption of server-side rendering (SSR) in the new stack ensures faster initial page loads and better search engine optimization (SEO).
• AI agents like Coder can effectively assist in debugging and migrating code within the Convex environment, leveraging its structured and synchronized data model.

Resources

• Thumbfa.st (Application): https://thumbfa.st
• NewStack Website: https://codeleyn.com/nowstack
• Convex Official Documentation: https://www.convex.dev
• TanStack Official Documentation: https://tanstack.com
• Next.js Official Documentation: https://nextjs.org
• Prisma Official Documentation: https://www.prisma.io
• Qstash Official Documentation: https://qstash.dev
• Excalidraw (Diagramming Tool): https://app.excalidraw.com
• GitHub (for code changes/PRs): https://github.com