Claymation Foley Planner
Lovable AI plans the quirky physical movements of claymation so animators hear their characters; ElevenLabs generates the SFX.
Generative Music & SFX· audio generation
Section · Voice
full primer →The primitive.
Videographers type a short prompt and ElevenLabs conjures a fitting sound effect or music bed for stop-motion sound design on demand, ready to drop into the timeline.
Why this primitiveSFX generation creates the specific, tactile, and slightly exaggerated squishes and steps needed for stop-motion animation.
Kernel
ElevenLabs /v1/sound-generation (≤2022s SFX) and /v1/music (studio tracks) — server takes a text prompt and returns an MP3 the client plays or downloads
Drives the UI as
a prompt box that conjures a sound effect, sting, or short track on demand and pipes it into the workspace
Required key.
ELEVENLABS_API_KEY
Single key for TTS, voice agents, scribe, music and SFX. Free tier covers a hackathon weekend.
open ↗Add this in your Lovable project under Settings → Secrets before pasting the prompt below.
Appendix · Mega-prompt
The build prompt.
budget · 1 message
Paste into a fresh Lovable project. Make sure the key above is set first. read the build strategy →
Build "Claymation Foley Planner" as a ONE-SHOT Lovable build. The participant has only
5 credits — this single message must produce a working demo with no follow-ups.
Single-page TanStack Start app. Cut scope ruthlessly.
CONCEPT
Lovable AI plans the quirky physical movements of claymation so animators hear their characters; ElevenLabs generates the SFX.
Discipline: Videography & Film (stop-motion sound design).
Recipe: Lovable AI brain + Generative Music & SFX (audio generation) as the voice surface.
Why this voice surface: SFX generation creates the specific, tactile, and slightly exaggerated squishes and steps needed for stop-motion animation.
LOVABLE BUDGET (HARD CAP: ONE-SHOT, ~5 CREDITS TOTAL):
The participant has FIVE Lovable credits for the whole build. This prompt MUST
ship a working demo on the FIRST message with zero follow-ups. Engineer for that.
- ONE TanStack Start app, ONE route (`src/routes/index.tsx`). No extra pages, no auth, no nav.
- TWO TanStack server functions max: one for the Lovable AI call (the brain),
one for the ElevenLabs call (the voice). Fold them into one if the voice
primitive does not need server-side text generation.
- ONE client surface (a button, a mic, or a prompt box) wired to those server fns.
- NO database, NO Lovable Cloud, NO auth, NO file uploads, NO extra integrations.
- NO tests, NO docs pages, NO settings screens, NO theming toggles.
- Libraries: template defaults + `ai` + `@ai-sdk/openai-compatible` + `zod`,
plus (only if the voice kernel needs it) `@elevenlabs/react`. Nothing else.
- Keep the diff small enough to land in one build pass. If a feature is not on
screen in the user flow below, do not build it. Cut scope before adding scope.
STACK
- TanStack Start app, the index route only.
- Lovable AI Gateway (the brain) + ElevenLabs (the voice). All calls live
inside `createServerFn` handlers so both keys stay on the server.
- Client surface fits the kernel: a prompt box for TTS / music / SFX, a mic
for agents, a live caption strip for scribe. Render markdown from the brain
with `react-markdown` if you show the text on screen.
- Tailwind + shadcn. Editorial look: gold accent on a dark or warm-cream
background, generous type, one strong headline, one primary action.
- Footer renders: "Built during the Creative AI & Quantum Hackathon organised by StreetKode Fam during Indian Krump Festival 14".
BRAIN — Lovable AI Gateway (free for participants, no key prompt needed):
```ts
// src/lib/ai-gateway.server.ts
import { createOpenAICompatible } from "@ai-sdk/openai-compatible";
export function gateway() {
return createOpenAICompatible({
name: "lovable",
baseURL: "https://ai.gateway.lovable.dev/v1",
headers: {
"Lovable-API-Key": process.env.LOVABLE_API_KEY!,
"X-Lovable-AIG-SDK": "vercel-ai-sdk",
},
});
}
```
Default model: `google/gemini-3-flash-preview`. Use `generateText` (or
`streamText` for long output) from `ai`. Keep prompts and model calls inside
the server function — never call the gateway from client code.
SERVER FUNCTION (src/lib/score.functions.ts) — brain writes the prompt, voice generates the audio:
```ts
import { createServerFn } from "@tanstack/react-start";
import { generateText } from "ai";
import { z } from "zod";
import { gateway } from "./ai-gateway.server";
/** Built during the Creative AI & Quantum Hackathon organised by StreetKode Fam during Indian Krump Festival 14 */
export const score = createServerFn({ method: "POST" })
.inputValidator((d) => z.object({
intent: z.string().min(1).max(500),
kind: z.enum(["sfx", "music"]).default("music"),
}).parse(d))
.handler(async ({ data }) => {
// 1. BRAIN — Lovable AI turns a vague creative intent into a strong
// ElevenLabs prompt (genre, mood, instruments, BPM, key, structure).
const { text: musicPrompt } = await generateText({
model: gateway()("google/gemini-3-flash-preview"),
system: `You are a music director for stop-motion sound design. Given the user's intent, ` +
`write a single-paragraph ElevenLabs prompt with genre, mood, ` +
`instrumentation, BPM, key, and structure. No preamble.`,
prompt: data.intent,
});
// 2. VOICE — ElevenLabs renders the audio.
const endpoint = data.kind === "music"
? "https://api.elevenlabs.io/v1/music"
: "https://api.elevenlabs.io/v1/sound-generation";
const body = data.kind === "music"
? { prompt: musicPrompt, duration_seconds: 30 }
: { text: musicPrompt, duration_seconds: 5, prompt_influence: 0.3 };
const r = await fetch(endpoint, {
method: "POST",
headers: { "xi-api-key": process.env.ELEVENLABS_API_KEY!, "Content-Type": "application/json" },
body: JSON.stringify(body),
});
const buf = await r.arrayBuffer();
return { musicPrompt, audio: Buffer.from(buf).toString("base64") };
});
```
CLIENT: input the creative intent, await `score()`, show the AI-written
`musicPrompt` so the user understands the choice, then play the audio data URI.
TRANSLATION — skip:
This kernel is non-linguistic (audio in or audio out), so do not add a
language selector or a translation pass. Keep the brain-to-voice path direct.
USER FLOW (the entire app — nothing else exists)
1. Land on the page; the headline previews what the demo does for stop-motion sound design.
2. The primary action (a prompt box that conjures a sound effect, sting, or short track on demand and pipes it into the workspace) is one tap away; the rest of the layout supports it.
3. Lovable AI does the thinking, ElevenLabs handles the voice surface, and the
result (audio + any text) stays on screen so the user can retry or share.
KEYS — both already provided to participants for free:
1. `LOVABLE_API_KEY` (the AI brain). Auto-injected in every Lovable project.
Read it only on the server via `process.env.LOVABLE_API_KEY`. Never prefix
with `VITE_` and never expose to the client.
2. `ELEVENLABS_API_KEY` (the voice). Best path: ask Lovable to connect the
ElevenLabs standard connector — it syncs the key automatically and rotates
from the dashboard. Fallback: paste a key from
https://elevenlabs.io/app/settings/api-keys into Project Settings -> Secrets.
Read it only on the server, header `xi-api-key`, never prefix with `VITE_`.
CREDIT (must appear in UI footer AND as JSDoc on the server function):
Built during the Creative AI & Quantum Hackathon organised by StreetKode Fam during Indian Krump Festival 14
Market sizing.
TAM
$1.1B
global video editing software market
SAM
$120M
animation and stop-motion post-production audio
SOM
$20M
indie animator and student filmmaker foley tools
Indicative figures for hackathon pitches — refine with your own research before raising.
Adjacent entries.
foley drafting
Foley Prompt Crafter
Lovable AI writes foley prompts for silent scenes so editors hear their vision; ElevenLabs generates the SFX.
temp scoringRough Cut Scorer
Lovable AI writes emotional beat sheets for rough cuts so directors feel the story; ElevenLabs scores it.
ambience layeringIndie Room Toner
Lovable AI designs room tone profiles for indie sets so audio feels grounded; ElevenLabs generates the SFX.
trailer sound designTrailer Riser Builder
Lovable AI plans trailer pacing and hit points so editors build anticipation; ElevenLabs generates the SFX.