← All work

UCL MSc RESEARCH / COMPUTATIONAL CREATIVITY

Evolving LLM prompts
for creative problem solving.

Can we systematically search prompt styles that encourage both useful and varied solutions? My MSc research combines a structured prompt grammar, MAP-Elites and two evaluation objectives to investigate this question.

CONTEXT
UCL MSc Computational Statistics and Machine Learning · 2025
MY WORK
Research design, Python framework, experiments and analysis
RESEARCH FOCUS
Evolutionary prompt search · Quality–diversity trade-offs

WHAT THE PROJECT INVESTIGATES

Search for prompt families.
Understand the trade-offs.

The task is resource-constrained, open-ended problem solving from the MacGyver dataset. The object of search is the prompt, while the outcome is the LLM’s proposed solution.

RQ1

Can evolutionary search uncover a diverse set of useful prompts?

Use multi-objective MAP-Elites to explore prompt styles while preserving solution-quality and response-variation trade-offs.

RQ2

Which prompt characteristics are associated with stronger scores?

Compare discovered prompt families, inspect component patterns and examine their relationship to the two evaluation objectives.

CONVERGENT QUALITY

Is the proposed solution useful?

One evaluator scores safety, feasibility and effectiveness on a 1–5 rubric.

Q = (safety × feasibility × effectiveness)^(1/3)
DIVERGENT VARIATION

How varied are the first-step responses?

Rephrased first steps are grouped by mutual NLI entailment. Entropy measures the spread of those groups.

H = −Σ p(cluster) log₂ p(cluster)

These are operational proxies for aspects of creativity. Response variation, prompt-style diversity and solution quality are different quantities.

3fixed problems, one per category
20search generations per archived run
2baselines: no added prompt and random CFG
Study configuration and prompt representation

The grammar has five components: role, thinking, creativity, combination and format instructions. I proposed seed ideas and used a generative model to expand the grammar, as described in the thesis. Search remains bounded by this curated design.

Thesis Table 4.1 reports 1,500 UMAP training prompts, five solution variations, an NLI threshold of 0.6 and a 5×5 archive. The saved repository archives use 10×10 coordinates; the separate viewer below explicitly re-bins them to a 5×5 display.

Each static baseline contains 20 outputs per task. The search archive contains selected candidates from a different sampling budget. The comparison is descriptive, without a matched-budget causal estimate.

SYSTEM ARCHITECTURE & ALGORITHM DESIGN

A search loop with
inspectable decisions.

Preparation defines the prompt space. The loop links generation, two-objective evaluation and a style-indexed archive. Logging makes the resulting search available for analysis.

PREPARATION
MacGyver tasksProblems + available tools + constraints
Prompt grammarFive instruction components
Fixed style projectionSample prompts → embeddings → fit UMAP

The evolutionary search loop

Select a module to inspect its design and implementation.

Select an archived parent → mutate its prompt components → repeat

The prompt feeds both generation and style mapping. Scores and style coordinates meet at archive update; the archive supplies parents for the next generation.

01 / COMPOSE A PROMPT

A structured search space for prompt styles.

The context-free grammar combines role, thinking, creativity, combination and format instructions. MAP-Elites selects a parent from its archive and mutates grammar components before constructing the next prompt.

INPUT
Task statement + grammar genotype
OUTPUT
Full prompt text + prompt-style components
MODEL-SERVICE ROLES

Four roles behind one HTTP adapter.

GeneratorSolutions + first-step variations
EvaluatorOne judge, three rubric scores
EmbeddingPrompt-style representations
NLIMutual entailment for variation clusters
LOGGING & ANALYSIS

Archive snapshots + JSONL recordsBaseline comparisons · style clusters · statistical tests · qualitative review

Read the complete data flow
Source-audited module contracts. Arrows describe data dependencies.
ModuleInputOutput
Compose a promptTask statement + grammar genotypeFull prompt text + prompt-style components
Generate a solutionFull candidate promptGenerated solution text
Evaluate two objectivesProblem + generated solutionQuality score, variation score and rubric scores
Locate the prompt styleRole, format and creativity instructionsTwo-dimensional style descriptor and cell address
Update the archiveStyle cell + scored candidateUpdated local Pareto set and parents for mutation

RESEARCH FINDINGS

Creativity involves a balance.

The thesis reports different search dynamics across tasks. The Outdoors result makes the central trade-off particularly clear: variation can increase while quality remains below a simpler baseline.

ORIGINAL THESIS FIGURESChapter 4 · Reported results
Thesis Figure 4.1: MAP-Elites versus no-prompt and random-prompt baselines for Household; quality on the left and response variation on the right.
Figure 4.1 · HouseholdQuality and variation rise together in this reported run.PDF page 42 · Left: convergent quality. Right: divergent variation. Original pixels retained.
01 / SEARCH

Preserve several strategies.

The archive retains alternatives across prompt styles and their quality–variation trade-offs.

02 / EVALUATION

Keep the objectives visible.

A more varied response does not automatically solve the task more safely or effectively.

03 / SCOPE

Interpret results within the study.

Three fixed problems and a single saved search per problem support case-specific findings.

FROM SEARCH TO ANALYSIS

Which prompt styles
go with which outcomes?

The second research question examines discovered prompt families through UMAP clustering, score comparisons, Welch tests and exploratory PCA. The figures below are from the thesis’s Household analysis.

Original thesis Figure 4.7, Prompt-style clusters, Household task.
Thesis Figure 4.7 · PDF page 50. Open original ↗

EXPLORATORY ANALYSIS

Different prompt families,
different score profiles.

The thesis groups discovered prompt styles, compares their score distributions and examines prompt-component patterns with PCA.

Household clusters · Thesis Table 4.2
ClusterQualityVariationn
C03.840.5424
C13.220.3822
C23.850.7647
C34.400.486
C44.000.999

C3 has the highest reported quality mean, based on six samples; C4 has the highest variation mean. These are discovered groups, without a controlled causal ablation.

Statistical method and interpretation

The thesis uses pairwise Welch tests to compare cluster means. For Household quality, C1 versus C3 reports t = −3.13 and p = 0.0080 (Table 4.3). This is an unadjusted, exploratory comparison among search-selected outputs.

The audited PCA code factorises categories separately within clusters before averaging their codes. This limits comparability and interpretation of component loadings. A stronger follow-up would use a common categorical encoding, correction for multiple comparisons and independent repeated runs.

Cluster separation and PCA loadings do not establish that a prompt component causes a change in creativity.

INSPECT A SAVED EXPERIMENT

Follow a prompt through to its output.

Open the offline explorer to inspect real prompt text, model responses, rubric scores and five saved snapshots for each of the three tasks.

A separate view of the archived data.

The thesis figures above are reproduced from the submitted document. This viewer uses repository archives and recomputed means. Its 5×5 map is a labelled re-binning of saved 10×10 coordinates; it is not the thesis’s original search grid.

Open the archived-results explorer
THE RESULTS WORKBENCH

Inspect the evidence.

Saved results · No model calls
Read the benchmark task MacGyver #358

Your puppy chewed up a pillow, and now there are feathers everywhere. You don't have a vacuum cleaner available. Tools available include a yoga ball, a fan, a broom with a short, broken handle, a lint roller, and a roll of weak-adhesive packing tape. The tape can't pick up the feathers and the broom is difficult to use. How will you clean up the feathers using only these items?

40 archived candidates · 15/25 display bins occupied

Prompt landscape

5 × 5 display view
Style projection · Y
Style projection · X
Lower meanHigher mean

Numbers count saved candidates. Select a bin to inspect its outputs. Arrow keys move focus; Enter selects.

Re-binned from saved style projections for this 5×5 view. These are display coordinates, not the original search cells or quality axes.

BIN 0 / 1

Follow the output.

1 / 5
Evaluator quality3.56 / 5
Response variation0.95 bits
Safety5/5Feasibility3/5Effectiveness3/5
Created at generation 18
View the exact prompt

You are a teacher in a rural school. Think as if you are solving the problem in a dream and in reality at the same time. Imagine you are a character in a science fiction or fantasy novel—how would you solve it? Hybridize two different problem-solving strategies. Write your solution in a step-by-step list with 2 steps. Here is the Problem: Your puppy chewed up a pillow, and now there are feathers everywhere. You don't have a vacuum cleaner available. Tools available include a yoga ball, a fan, a broom with a short, broken handle, a lint roller, and a roll of weak-adhesive packing tape. The tape can't pick up the feathers and the broom is difficult to use. How will you clean up the feathers using only these items?

ARCHIVED MODEL OUTPUTUnverified response
**The Dream Reality: "Feather Frenzy Solution"**

As a teacher in a rural school, I've encountered my fair share of creative problems. In this hybrid dream-reality scenario, I'll combine elements of both the dream world and the constraints of the rural school setting to devise a step-by-step solution to the feathered mess.

**Step 1: "Feather Storm" Taming**
I'll use the yoga ball and the fan to create a makeshift air current that will help me corral the feathers. I'll roll the yoga ball across the room, using it as a makeshift "feather-sweeper." As the ball rolls, I'll use the fan to blow the feathers in the direction of the ball. This will create a mini-storm that will help gather the feathers in a more manageable area. The broken handle on the broom can be used as an impromptu feather "collector" to temporarily catch stray feathers.

**Step 2: "Lint Roller Lasso"**
Now that the feathers are more contained, I'll utilize the lint roller to pick up the feathers. I'll use the weak-adhesive packing tape to attach the lint roller to the broken handle of the broom, effectively creating a makeshift "lint roller lasso." By holding the lasso close to the feathers, I can carefully pick up the individual feathers, gradually reducing the amount of debris in the room. Once I've collected a decent amount of feathers, I can repeat the process until the room is tidy.

**The Dream Reality Method:**

*   **Lucid problem-solving**: In this hybrid scenario, I'll tap into my dream state to envision creative solutions and connect seemingly unrelated ideas.
*   **Rural resourcefulness**: Drawing from my experience as a teacher in a rural school, I'll rely on the available tools to devise an innovative solution that leverages the strengths of each item.

**Step-by-Step Solution:**

1.  **Feather Storm Taming**: Use the yoga ball and fan to corral the feathers, and the broken handle on the broom to catch stray feathers.
2.  **Lint Roller Lasso**: Attach the lint roller to the broken handle of the broom using weak-adhesive packing tape and use it to pick up individual feathers.

By combining elements of both the dream world and the constraints of the rural school setting, I've devised a creative solution to the feathered mess. With the right mindset and resources, even the most seemingly insurmountable problems can

Candidate text and scores are unchanged. No model requests, search reruns or invented intermediate snapshots are used.

RESEARCH CONTRIBUTION & LIMITS

The contribution,
and what remains open.

Method integration

Combine a structured prompt grammar, quality–diversity search and explicit evaluation into one research framework.

Experimental engineering

Implement model-service adapters, style-embedding reuse, archive updates and persistent experiment records in Python.

Empirical analysis

Connect baseline trajectories, discovered prompt groups and qualitative examples, while examining the limits of the metrics.

What this study can support

The work studies three fixed problems, with limited model-call budget and no independent repeated search seeds. The evaluator is one LLM judge, and first-step entropy is a proxy. These conditions limit generalisation and statistical certainty.

Exploratory comparisons among search-selected outputs are not independent replications. A follow-up should include more tasks, repeated seeds, matched evaluation budgets and human-calibrated scoring, with uncertainty estimates and multiple-comparison controls.

Thesis and implementation distinctions

The thesis describes a 5×5 grid; the saved archives use 10×10 coordinates. The confirmed portfolio view re-bins saved style coordinates into fixed 5×5 display bins for each task, preserving all candidates. Display occupancy is not original archive coverage.

Several thesis trajectories differ from the saved archives. For the Household final archive, candidate quality averages to 4.178, while its evolution log records 4.278. The offline chart uses candidate-derived values. The original thesis figures remain separate.

The audited MAP-Elites route mutates archived genotypes. Crossover belongs to a separate genetic-algorithm route. UMAP preparation embeds complete prompts, while the search descriptor embeds role, format and creativity instructions. The map axes are style coordinates, not objective scores.

The current PCA implementation uses cluster-local categorical codes, which limits its interpretation. Neither its loadings nor exploratory cluster comparisons establish causal effects of prompt components.

Established methods and my research role

This is my individual MSc research project, supervised by Peter J. Bentley. My contribution is the research formulation, framework implementation, experiment design and analysis. MAP-Elites, UMAP, non-dominated sorting and NLI are established methods used in the study.

The grammar incorporates my seed designs expanded with LLM assistance. Four model-service roles do not constitute an autonomous multi-agent system. The project is presented as evolutionary algorithm research and API-based experimental engineering.

Evidence and reproducibility

The figures come from the submitted thesis, with original pixel values retained. Archive data is pinned to commit 6926fbdd824c, and each snapshot is validated before export. The new website interface is a portfolio adaptation.

The fixed source snapshot lacks an imported analysis helper and a locked environment. A fresh end-to-end execution is not verified. No research entry point, model API or saved pickle is executed by this page.

Search implementation ↗Evaluation implementation ↗Analysis implementation ↗

THESIS

Illuminating Prompt Space for Discovering Creativity in LLM with MAP-Elites

Richard Huang · MSc Computational Statistics and Machine Learning, UCL.
Supervised by Peter J. Bentley · Submitted 12 September 2025.

APPLIED ML & RESEARCH ENGINEERING

Have a modelling question worth investigating?

Bring the problem, the available evidence and the decision the work needs to support.

Discuss an ML Project