Agents using real desktop apps

FreeCAD 0.21 3D CAD

FreeCAD is already open with ~/Documents/plate.FCStd loaded; it holds one part, Plate, a 160 x 100 x 5 mm box. Using FreeCAD's menus, toolbars and dialogs only (not the Python console or macros), cut a 10 mm diameter hole straight through the middle of the plate: switch to the Part workbench (View > Workbench > Part), add a cylinder from it and name it Hole, size and place it with its properties, subtract it from Plate with Part > Boolean > Cut, then save the document in place.

Graded by Document.xml inside the saved .FCStd: a Part::Cut, and a 5 mm radius cylinder at the plate's centre

KiCad 7 Schematic Editor

In the USB LED indicator schematic open in KiCad's Schematic Editor, change resistor R1's value from 330 to 220 (ohms), then save the schematic.

Graded by R1's value in the saved .kicad_sch schematic

QElectroTech 0.9 electrical diagram editor

Request “Build me a QElectroTech environment where the agent changes a component's label in a small electrical diagram and saves it.”

In QElectroTech, change the label of the contactor coil K1 in the diagram Documents/motor-starter.qet to KM1 and save it.

Graded by the saved .qet: the coil's new label, its function kept, three components

KmPlot 23.08 function plotter

Request “Build me a KmPlot environment where the agent changes the formula of one plotted function and saves the plot file.”

In KmPlot, change the formula of the function h in Documents/lesson_plots.fkt to h(x) = 2x + 1 and save the plot file.

Graded by the saved .fkt: h's new equation, the old one gone, f and g kept, three functions

LibreCAD 2.2 2D CAD

Add four more mounting holes to the plate drawing in LibreCAD and save it.

Graded by the circle count in the saved DXF drawing

LibreOffice Writer 24.2 word processor

In the delivery notice open in LibreOffice Writer, move the delivery to Monday 24 November (same time, 8am to 10am) by changing the date under the title, then save the notice.

Graded by content.xml inside the saved .odt: the subtitle's new date, title and signature untouched

AbiWord 3.0 word processor

Request “Build me an AbiWord environment where the agent changes the meeting time in a memo and saves it.”

The book sale planning meeting has moved. Change the meeting time in the memo Documents/book-sale-memo.abw to 2:30 PM and save it.

Graded by the saved .abw: the meeting paragraph's whole text, the agenda deadline kept

LyX 2.4 document processor

Request “Build me a LyX environment where the agent fixes a word in a section heading of a short document and saves it.”

In LyX, the heading of section 2 in Documents/garden-report.lyx says "Soil Preperation". Correct the misspelled word so the heading reads "Soil Preparation", and save the document.

Graded by the saved .lyx: the corrected heading, the typo gone, three sections, the other headings kept

JabRef 3.8 reference manager

Request “Build me a JabRef environment where the agent corrects a year in a small bibliography and saves it.”

In JabRef, correct the year of the Okafor and Lindqvist article "Seasonal Sediment Transport in Glacial Rivers" in Documents/references.bib to 2019 and save it.

Graded by the saved .bib: the 2019 year line, no 2017 line, the article's journal and volume kept

LibreOffice Calc 24.2 spreadsheet

Request “Build me a LibreOffice Calc environment where the agent fixes one month's figure in a small quarterly sales spreadsheet and saves it.”

February's sales figure in my quarterly sales spreadsheet, Documents/q1-sales.ods, is wrong: it should be 23,100. Fix it and save the spreadsheet.

Graded by content.xml inside the saved .ods: the sheet's whole text, with February's new figure and the recalculated total

Gnumeric 1.12 spreadsheet

Request “Build me a Gnumeric environment where the agent updates one month's figure in a small household budget spreadsheet and saves it.”

In my household budget spreadsheet, Documents/household-budget.xml, change the Groceries figure for March to 437.80 and save it.

Graded by the saved spreadsheet: March Groceries' new value, three neighbouring cells unchanged

LibreOffice Impress 24.2 presentation app

Request “Build me a LibreOffice Impress environment where the agent changes a slide's title in a small presentation and saves it.”

In the presentation Documents/garden-meeting.odp, change the title of the second slide from Planting Schedule to Spring Planting Calendar, and save it.

Graded by content.xml inside the saved .odp: the slide's new title, its bullets kept, three slides

DB Browser for SQLite 3.12 database editor

Request “Build me a DB Browser for SQLite environment where the agent corrects a customer's email address in a small contacts database and saves it.”

In DB Browser for SQLite, the customer contacts database Documents/contacts.db has the wrong email address for Priya Raman. Correct her email address to priya.raman@example.com and save the change.

Graded by the saved database: Priya Raman's corrected email, the rest of her row, another customer and the row count

HomeBank 5.7 personal finance manager

Request “Build me a HomeBank environment where the agent records a new grocery expense in a checking account and saves the file.”

In HomeBank, record a new grocery expense in the Everyday Checking account of Documents/household.xhb: $64.18 paid to Fresh Market on November 4, 2026, category Groceries, memo "Weekly groceries". Then save the file.

Graded by the saved .xhb: the new transaction's amount, account, date, payee and category, the rent unchanged

KMyMoney 5.1 personal finance manager

Request “Build me a KMyMoney environment where the agent records a payment in a small household finance file and saves it.”

In KMyMoney, record a payment of $87.45 to Riverside Electric from the Household Checking account, dated November 3, 2026, in the Utilities category, in my household finance file Documents/household.kmy, then save the file.

Graded by the saved .kmy: the new payment's date, amount, payee and category, five transactions, the October payment kept

GnuCash 5.5 accounting app

Request “Build me a GnuCash environment where the agent records a grocery purchase in a small household book and saves it.”

In my household book (Documents/household.gnucash), record a grocery purchase: on November 3, 2026 I paid $63.48 from the Checking Account at Greenleaf Market. Enter it with the description Greenleaf Market and file it under Expenses:Groceries, then save the book.

Graded by the saved book: the new transaction's two splits and date, five transactions, Corner Grocer kept

Planner 0.14 project manager

Request “Build me a Planner environment where the agent adds a 'User testing' task to a small project plan and saves it.”

Add a 'User testing' task to my website launch project plan in Documents/website-launch.planner and save it.

Graded by the saved .planner: a task named User testing, the four existing tasks kept

Freeplane 1.7 mind-mapping app

Request “Build me a Freeplane environment where the agent adds a child topic to a small mind map and saves it.”

In my Weekend Trip mind map (Documents/weekend-trip.mm) in Freeplane, add a child topic called Water filter under Packing, and save the map.

Graded by the saved .mm: one Water filter topic, no leaf turned parent, Sleeping bags kept

VYM 2.6 mind-mapping app

Request “Build me a VYM environment where the agent adds a branch to a small mind map and saves it.”

In VYM, add a branch called Sunset boat tour under Agenda in my Team Offsite mind map (Documents/team-offsite.vym) and save it.

Graded by map.xml inside the saved .vym: the new branch under Agenda, three Agenda branches, two kept

TreeLine 3.1 outliner

Request “Build me a TreeLine environment where the agent adds a new book under one of the categories in a reading-list outline and saves it.”

In TreeLine, add the book The Silk Roads by Peter Frankopan, published in 2015, under the History category of my reading list in Documents/reading-list.trln, and save it.

Graded by the saved .trln: one more BOOK under History (details not checked), categories kept

Inkscape 1.2 vector graphics editor

In the bake sale poster open in Inkscape, change the headline from "Spring Bake Sale" to "Summer Bake Sale", then save the poster.

Graded by the saved SVG: the headline's new text, price and date untouched

Inkscape 1.2 vector graphics editor

Request “Build me an Inkscape environment where the agent changes a poster's headline and saves it.”

Change the poster's headline in Documents/poster.svg to Autumn Seed Exchange and save it.

Graded by the saved SVG: the headline's new text, the date untouched

MyPaint 2.0 painting app

Request “Build me a MyPaint environment where the agent renames a layer in a small drawing and saves it.”

In my drawing Documents/sunny-day.ora, rename the layer called Sketch to Crescent Moon and save it.

Graded by stack.xml inside the saved .ora: a layer named Crescent Moon, none named Sketch, the other layers kept

Pencil2D 0.6 animation app

Request “Build me a Pencil2D environment where the agent renames a layer in a small animation and saves it.”

In Pencil2D, rename the "Bitmap Layer" layer of my bouncing ball animation in Documents/bounce.pclx to Ball and save it.

Graded by main.xml inside the saved .pclx: layer 3's new name, the old name gone, Ground kept, two bitmap layers

Dia 0.98 diagram editor

Request “Build me a Dia environment where the agent renames a box in a small flowchart and saves it.”

In my order flowchart, Documents/order-flow.dia, rename the "Check stock" box to "Verify inventory" and save it.

Graded by the saved .dia: the box's new text, one box so named, four boxes, the other boxes' text kept

LibreOffice Draw 24.2 drawing app

Request “Build me a LibreOffice Draw environment where the agent relabels a box in a small diagram and saves it.”

In my order fulfillment diagram, Documents/order-flow.odg, relabel the "Pick Items" box to "Pick and Scan" and save it.

Graded by content.xml inside the saved .odg: the box's new label, its neighbours kept, four boxes

Scribus 1.6 desktop publishing app

Request “Build me a Scribus environment where the agent changes the headline on a one-page event flyer and saves it.”

Change the headline of the event flyer in Documents/jazz-night-flyer.sla to Harbor Lights Jazz Festival and save it.

Graded by the saved .sla: the headline frame's new text, the details frame unchanged

Sigil 2.0 EPUB editor

Request “Build me a Sigil environment where the agent fixes a word in a chapter title of a short ebook and saves it.”

In Sigil, the title of chapter two in my ebook Documents/lantern-keeper.epub misspells "Mainland" as "Mainlnad". Fix that word in the chapter title and save the book.

Graded by the saved .epub: chapter two's corrected title, its text and the other titles kept

gLabels 3.4 label designer

Request “Build me a gLabels environment where the agent fixes the name on a small address label and saves it.”

The name on my address label in Documents/address-label.glabels is misspelled. It should read Marisol Penhallow. Fix the name in gLabels and save the label.

Graded by the saved .glabels: the label's whole text, with the corrected name and the address unchanged

MuseScore 3.2 music notation editor

Request “Build me a MuseScore environment where the agent changes the title and composer of a short score and saves it.”

In MuseScore, change the title of the score in Documents/MuseScore3/Scores/Morning_Song.mscz to Evening Lullaby and its composer to Clara Hollis, then save it.

Graded by the saved score: the title frame's whole text, with the new title and composer

Rosegarden 22.12 music sequencer

Request “Build me a Rosegarden environment where the agent renames a track in a small music project and saves it.”

In Rosegarden, rename the track called Bass to Bass Guitar in my Sunday jam project, Documents/sunday-jam.rg, and save it.

Graded by the saved .rg: track 1's new label, two neighbouring tracks kept, 32 tracks

Hydrogen 1.2 (beta) drum machine

Request “Build me a Hydrogen environment where the agent changes a song's tempo in a small drum pattern project and saves it.”

In Hydrogen, change the tempo of the Backbeat Sketch song in Documents/backbeat.h2song to 112 BPM and save it.

Graded by the saved .h2song: the new tempo, the song's name and the groove's notes kept

Tiled 1.8 map editor

Request “Build me a Tiled environment where the agent renames an object in a small tile map and saves it.”

In my Tiled map Documents/harbor_town.tmx, rename the object called Old Well to Wishing Well and save the map.

Graded by the saved .tmx: object 1's new name, the old name gone, two neighbours kept, four objects

Geany 2.0 code editor

In the shop's settings.py open in Geany, turn debug mode off and change the port to 8080, then save the file.

Graded by the saved settings.py: new lines present, old ones gone, rest untouched

Glade 3.40 GTK interface designer

Request “Build me a Glade environment where the agent changes a button's label in a small dialog design and saves it.”

In Glade, change the label of the Submit button in the feedback dialog design in Documents/feedback.glade to Send Feedback and save it.

Graded by the saved .glade: Submit's new label, Cancel kept, two buttons

Poedit 3.4 translation editor

Request “Build me a Poedit environment where the agent fixes a wrong translation in a French .po file and saves it.”

In Poedit, the French translation of "expected one argument" in Documents/fr.po is wrong. Fix it so it reads "un argument attendu", and save the file.

Graded by the saved .po: the corrected translation, the wrong one gone, five other translations kept

Umbrello 2.38 UML modeller

Request “Build me an Umbrello environment where the agent adds an attribute to a class in a small UML model and saves it.”

In Umbrello, add an attribute named email of type string to the Member class in the library model Documents/library.xmi and save it.

Graded by the saved .xmi: an email attribute of type string on Member, three attributes, the others kept

Mousepad 0.6 text editor

Add buy milk as a new line at the end of the to-do list open in Mousepad, then save it.

Graded by the saved list: buy milk added exactly once

ghostwriter 23.08 Markdown editor

Request “Build me a ghostwriter environment where the agent adds an item to a Markdown checklist and saves it.”

In ghostwriter, add Water filter to the Gear checklist in Documents/camping-trip.md and save it.

Graded by the saved .md: the new unticked item, once, the four gear items kept

Gnote 46 note-taking app

Request “Build me a Gnote environment where the agent adds olive oil to the shopping list note.”

Add olive oil to my Shopping List note in Gnote.

Graded by the saved note: olive oil added, two existing items kept

CherryTree 1.1 hierarchical note-taking app

Request “Build me a CherryTree environment where the agent adds a line to a note in a small notebook and saves it.”

In CherryTree, add a line saying Clean the gutters to the Weekend chores note in my notebook Documents/home-notebook.ctd and save it.

Graded by the saved .ctd: the new line in Weekend chores, the four chores kept, three notes

RedNotebook 2.32 journal

Request “Build me a RedNotebook environment where the agent adds a sentence to an existing day's journal entry and saves the journal.”

In RedNotebook, add the sentence "Moved the rosemary to the sunny corner by the railing." to the end of my journal entry for Tuesday, 20 October 2026, then save the journal.

Graded by the saved month file: the new sentence after the entry's last words, the entry and another day kept

Xournal++ 1.2 handwriting notebook

Request “Build me a Xournal++ environment where the agent adds a text note to a small notebook page and saves it.”

In Xournal++, add a text note that says "Quiz on Friday: bring a calculator" to the Cell Biology notebook page in Documents/biology-notes.xopp, and save it.

Graded by the saved .xopp: the new note's text, four notes in the layer, the original three kept

Zim 0.75 desktop wiki

Request “Build me a Zim environment where the agent adds an action item to a meeting-notes page.”

In my Zim notebook, add a new action item to the Meeting Notes page: "Dana to update the release checklist". Add it as an unchecked checkbox in the Action items list, like the others there, and save the page.

Graded by the saved page: the new unchecked action item, once, the other items kept

Osmo 0.4 personal organizer

Request “Build me an Osmo environment where the agent adds a new contact to the address book.”

In Osmo, add a new contact to my address book: Delphine Moreau, cell phone 555-0163, e-mail delphine.moreau@example.com.

Graded by the saved address book: the new contact's name, phone and e-mail, an existing contact kept

Tellico 3.5 collection manager

Request “Build me a Tellico environment where the agent adds a book to a small book collection and saves it.”

Add the book The Glass Orchard by Aurelio Benn to my book collection in Documents/my-books.tc, with Northwind House as the publisher and 2022 as the publication year, and save it.

Graded by the saved .tc: the new book's title, author, publisher and year, two existing books kept

Parley 23.08 vocabulary trainer

Request “Build me a Parley environment where the agent adds a new word pair to a Spanish vocabulary collection and saves it.”

In Parley, add a new word pair to my Spanish Basics vocabulary collection in Documents/spanish.kvtml: the English word "library" with the Spanish "biblioteca". Then save the collection.

Graded by the saved .kvtml: the library/biblioteca pair, seven entries, two pairs kept