Compendium

Survey / Questionnaire / Experiment DSLs Family Index

13 min read

Survey / Questionnaire / Experiment DSLs Family Index

type: language-family-index family: survey-questionnaire languages_catalogued: 24 tags: [language-reference, family-index, survey-questionnaire, qualtrics, redcap, odk, xlsform, kobotoolbox, surveyjs, jspsych, psychopy, opensesame, eprime, cdisc-odm]

Survey / Questionnaire / Experiment — Family Index

Family overview

Survey, questionnaire, and experiment DSLs are the textual languages used to describe instruments — the questions, branching logic, validation rules, randomisation schedules, and stimulus timings that elicit data from human respondents. Unlike general-purpose forms (HTML form, JSON Schema), these languages encode domain-specific concerns the general web stack does not address: branching logic (“show Q5 only if Q3 == ‘yes’”), piped text (substitute Q1’s response into Q4’s stem), randomisation (counterbalancing, Latin squares, blocked random), multilingual delivery (one canonical question with locale-tagged variants), scoring rubrics (Likert summation, sub-scale composites), trial timing (millisecond-accurate stimulus onset, response windows), and regulatory traceability (audit trails, electronic signatures, 21 CFR Part 11 in clinical-trial contexts).

The ecosystem splits into four loosely overlapping worlds. The academic / clinical research world is dominated by two giants: Qualtrics (proprietary, dominant in higher-education and corporate UX research; .qsf is its JSON-based survey export) and REDCap (Vanderbilt, 2004; the dominant academic clinical-research tool, with the CSV-based “data dictionary” as its canonical instrument definition; ~7,000+ institutional partners as of 2026 and currently on the 17.x line, with v17.0.2 released April 2026). The mobile / humanitarian data collection stack is the ODK / XLSForm / KoBoToolbox lineage — XForms (a W3C subset) as the wire format, XLSForm as the spreadsheet authoring shorthand, ODK Collect / KoboCollect as the Android client, and KoBoToolbox as the humanitarian-sector hosted instance (UN OCHA, MSF, Red Cross workflows).

The web survey schema layer is where SurveyJS (open-source JS form-builder library, currently on 2.5.x with v3.0 targeted for 2026) sits alongside the proprietary SaaS schemas: Typeform, Tally, Forms.app, Google Forms, Microsoft Forms. These are JSON-based, web-first, and increasingly converge toward JSON-Schema-derived shapes — but each remains incompatible at the wire level. The fourth world is psychology experiments, an entirely separate genealogy where millisecond timing, stimulus pre-loading, and randomisation are first-class concerns: jsPsych (browser-based JavaScript, 8.x line — v8.2.3 February 2026), PsychoPy (Python + visual Builder, releasing on a YYYY.x calendar with 2026.1.3 in March 2026), OpenSesame (Python-based GUI, 4.1 “Neonatal Nightingale”, 4.1.9 February 2026), and the proprietary stalwarts E-Prime 3 (Psychology Software Tools, Update 4 current) and NBS Presentation / Inquisit.

The clinical-trial CRF chain glues these worlds to regulators: OpenClinica (OC4 line, ongoing 2026 updates), REDCap (often used as a regulated EDC for academic trials), and CDISC ODM-XML (v2.0 released 2023, breaking backward compatibility with 1.3.2; the underlying interchange format that lets a CRF travel between EDC vendors and into FDA submissions) — see healthcare-clinical for the regulatory-side detail. The recent shift across all four worlds is toward JSON-Schema-derived definitions, OAuth / FHIR Questionnaire interoperability, and (in psychology) the Open Science Framework / pre-registration drive that pushes experiment scripts into public repositories with explicit hashes.

In our deep library

No survey/experiment DSL has its own deep-library note — these are configuration-style instrument languages on top of host languages, not general-purpose languages.

Cross-reference:

  • healthcare-clinical — CDISC ODM-XML, Define-XML, FHIR Questionnaire, and HL7 CDA all overlap with this family at the regulatory boundary; ODM-XML is dual-classified.
  • api-description — modern web survey schemas (SurveyJS, Typeform JSON, Tally) are increasingly JSON-Schema-derived; the line between “form schema” and “API schema” is thin.
  • notation-spec — formal grammar adjacent (the W3C XForms 1.0 spec is itself a formal language definition that ODK XForms subsets).
  • i18n-locale — multilingual surveys (REDCap Multi-Language Management, Qualtrics translations, ODK label::lang) use the same BCP-47 / ICU MessageFormat machinery covered there.
  • python — host language for PsychoPy, OpenSesame, the pyxform XLSForm→XForms compiler, and most REDCap API integrations (PyCap).
  • javascript — host language for jsPsych, lab.js, SurveyJS, and most modern web survey runtimes.

Tier 3 family table — Academic / clinical research forms

FormatFirst appearedOriginTypeStatus (2026)URL
Qualtrics .qsf (Survey File)mid-2000s (Qualtrics founded 2002, QSF format consolidated late 2000s)Qualtrics LLC (Provo, UT; SAP acquisition 2018, spun out 2021, taken private by Silver Lake 2023)JSON-based survey definition (single-document, with SurveyEntry + SurveyElements arrays); proprietary, undocumented officiallyVery active — Qualtrics XM remains the dominant higher-ed and corporate survey platform; QSF is the canonical export format in 2026https://www.qualtrics.com/support/survey-platform/survey-module/survey-tools/import-and-export-surveys/
Qualtrics QSF block JSON(internal to QSF)QualtricsSub-structure within QSF: Blocks and BlockElements describe page/block grouping, randomisation, and branchingActive (always shipped with QSF); reverse-engineered libraries like sumtxt/qsf (R) document the structurehttps://gist.github.com/ctesta01/d4255959dace01431fb90618d1e8c241
REDCap data dictionary2004 (REDCap founded, Vanderbilt; Paul Harris)Vanderbilt University Medical Center; consortium of 7,000+ institutions globallyCSV-based instrument definition: one row per field with field_name, form_name, field_type, field_label, choices, branching_logic, validationVery active — REDCap 17.0.2 (Apr 2026); 16.1.4 (Mar 2026) added new Smart Variables, Public Reports/Dashboard access codes, and AI-assisted multi-language managementhttps://projectredcap.org/
REDCap External Modules JSON config2017 (External Modules framework)Vanderbilt + UF (Mark McEver, Andy Martin)config.json schema for REDCap plugins; declares settings, system/project-level permissions, hooksVery active — the EM marketplace is the de facto extension layer for REDCaphttps://github.com/vanderbilt-redcap/external-module-framework-docs
OpenClinica CRF (XLS-CRF + ODM-XML)2005 (OpenClinica 1.0); XLS-CRF authoring layer ~2006+Akaza Research → OpenClinica LLCSpreadsheet-based CRF authoring in OC3; OC4 uses ODM-XML internally as the canonical exchange formatActive — OC4 receiving regular 2026 updates (Queries page performance, discrepancy notes fixes); OC3 in long-term maintenancehttps://docs.openclinica.com/oc4/
CDISC ODM-XML v2.01999 (ODM v1.0); v2.0 released 2023CDISC consortium (Clinical Data Interchange Standards Consortium)XML (and v2.0+ optionally JSON) operational-data interchange model; carries study definitions, CRFs, audit trails, signatures; FDA-recognisedActive — v2.0 is current; breaks backward compatibility with 1.3.2; foundation for Define-XML, SDM-XML, Dataset-XML, Dataset-JSON; cross-listed with healthcare-clinicalhttps://www.cdisc.org/standards/data-exchange/odm-xml/odm-v2-0
OCMP (Operational Clinical Manuscript)2010sNiche clinical-trial vendorsProprietary CRF specification format used in some EDC toolingNiche / vendor-specific; ODM-XML has subsumed most of the interoperability use case(vendor-specific)
ICH / WHO / NIDA CRF templates (CDASH-aligned)CDASH v1.0 2008; CDASHIG continuously updatedCDISC + ICH (International Council for Harmonisation)Reference CRFs and standard data-collection element libraries; mostly distributed as PDF + ODM-XMLActive — CDASHIG v2.3 era; widely used as starter templates in REDCap and OpenClinicahttps://www.cdisc.org/standards/foundational/cdash

Tier 3 family table — Mobile / humanitarian data collection

FormatFirst appearedOriginTypeStatus (2026)URL
W3C XForms 1.02003 (W3C Recommendation)W3C Forms Working GroupXML-based form description model (instance + binding + UI separation); MVC-styleLegacy at W3C (the WG closed years ago) but the subset lives on via ODK XForms; the canonical ancestor of the mobile-data-collection stackhttps://www.w3.org/TR/xforms/
ODK XForms2008 (Open Data Kit, U Washington / Carl Hartung, Yaw Anokwa)University of Washington, ODK community → Get ODK Inc.Subset of W3C XForms 1.0 with ODK-specific extensions (instance metadata, audit logging, geopoint/geotrace types, encrypted submissions)Very active — the wire format underneath ODK Central, Collect, KoBoToolbox, SurveyCTO, Ona, Enketohttps://getodk.github.io/xforms-spec/
XLSForm2010s (early; SurveyCTO + ODK community)XLSForm.org / community standard; pyxform reference compilerSpreadsheet shorthand: survey, choices, settings sheets compile to ODK XForms via pyxform; the practical authoring layer 95%+ of ODK users actually touchVery active — pyxform supports Python 3.10–3.13 (3.12 primary); de facto authoring standardhttps://xlsform.org/
ODK Central forms2018+ (ODK Central server)Get ODK Inc.Server-side form storage and submission API for ODK XForms; v2024+ supports OData feeds and entities (cross-form data references)Very active — the dominant self-hosted ODK back end as of 2026https://docs.getodk.org/central-intro/
KoBoToolbox forms2014 (Harvard Humanitarian Initiative; later spun out as KoBoToolbox Inc.)HHI / KoBoToolbox; widely deployed by UN OCHA, UNHCR, MSF, Red CrossODK-XForms-compatible with KoBo-specific extensions; XLSForm-authored; hosted at kf.kobotoolbox.org and self-hostable via kobo-installVery active — kobo-install 2.026.12d (May 2026); KoboCollect requires Android 8.0+ from 2026 onwardhttps://www.kobotoolbox.org/
Enketo / Enketo Express XForm runtime2012 (Martijn van de Rijdt)Enketo LLC; widely used as the web/JS runtime for ODK / KoBo / OpenClinica formsPure-JS renderer for ODK XForms (browser-based offline-capable); no separate format but a major XForms-runtime siblingActivehttps://github.com/enketo/enketo

Tier 3 family table — Web survey schemas (proprietary + open)

FormatFirst appearedOriginTypeStatus (2026)URL
SurveyJS schema2014 (Andrew Telnov)SurveyJS / Devsoft Baltic OÜOpen-source JSON-based survey schema; typed question library; runs against React/Angular/Vue/vanilla; the standalone Survey Creator is the visual editorVery active — v2.5.11 (Feb 2026); v3.0 targeted Q1 2026 with UI Preset Editor and Tailwind/Bootstrap adapters; major web open-source contenderhttps://surveyjs.io/
LimeSurvey LSS2003 (PHPSurveyor → LimeSurvey 2007)LimeSurvey GmbHXML survey export (.lss); structured-archive format .lsa for full survey + responsesActive — LimeSurvey 6.x line; widely used in academic settings as a self-hostable alternative to Qualtricshttps://www.limesurvey.org/
Typeform schema2012 (Typeform, Barcelona)Typeform SLProprietary JSON form schema exposed via Create API; conversational one-question-per-screen UXActive — closed schema but well-documented public REST APIhttps://www.typeform.com/developers/create/
Tally schema2020 (Tally.so, Belgium)TallyProprietary JSON schema; minimal-UX competitor to Typeform; freemiumActivehttps://tally.so/
Forms.app schema2018Forms.appProprietary JSON schema; SaaS form builderActivehttps://forms.app/
Google Forms schema2008Google WorkspaceInternal proprietary JSON; Forms API (GA 2022+) exposes a structured schema for read/write but no offline export formatActive but limited as a programmatic formathttps://developers.google.com/forms/api
Microsoft Forms / Forms Pro2016 (Forms); Forms Pro 2019 (rebranded Customer Voice 2020, refreshed within Dynamics 365)Microsoft 365Proprietary JSON; Graph API exposes limited schemaActive but consumer-grade; enterprise survey work increasingly routed through Dynamics 365 Customer Voicehttps://forms.office.com/
FHIR Questionnaire (R5)FHIR DSTU1 2014; R4 2019; R5 2023HL7 FHIRJSON / XML / Turtle questionnaire resource; cross-system clinical-questionnaire interchange standardActive — primary route for survey/CRF interoperability inside healthcare systems; cross-listed with healthcare-clinicalhttps://hl7.org/fhir/questionnaire.html

Tier 3 family table — Psychology experiments

FormatFirst appearedOriginTypeStatus (2026)URL
jsPsych experiment scripts2013 (Josh de Leeuw)Vassar College → Indiana / communityJavaScript library; experiments are JS arrays of “trial” objects describing plugins, parameters, timing; runs in the browserVery active — v8.2.3 (Feb 2026); v8.x is a complete core rewrite; the de facto online psychology frameworkhttps://www.jspsych.org/
JATOS + jsPsych integrationJATOS 2014 (Kristian Lange, MPIB Berlin)Max Planck Institute for Human DevelopmentSelf-hostable Java server for distributing jsPsych (and other JS) experiments; not a DSL itself but the dominant deployment substrateActivehttps://www.jatos.org/
lab.js / lab.js builder2015 (Felix Henninger)Open-source communityBrowser-based builder + JSON study description; targets behavioural / cognitive sciences with millisecond-accurate timingActive — published Behavior Research Methods 2021; visual builder + JS exporthttps://lab.js.org/
PsychoPy Builder XML2002 (Jon Peirce, Nottingham)University of Nottingham; community.psyexp XML format generated by Builder GUI; compiles to Python (or PsychoJS for online via Pavlovia)Very active — PsychoPy 2026.1.3 (Mar 2026); calendar-versioned (YYYY.minor.patch); 2026.2.0 expected autumn with Apple Silicon supporthttps://psychopy.org/
OpenSesame (.osexp)2010 (Sebastiaan Mathôt et al.)University of Aix-Marseille; Cogsci.nlVisual experiment builder; .osexp is a zipped archive containing a script + assets; Python scripting via inline_script itemsActive — 4.1 “Neonatal Nightingale”, 4.1.9 (Feb 2026)https://osdoc.cogsci.nl/
E-Prime experiment (.es / .es3 / .ebs)1996 (E-Prime 1.0)Psychology Software Tools (Pittsburgh)Proprietary experiment files; E-Basic scripting language; .ebs is the compiled binary, .es / .es3 the sourceActive — E-Prime 3.0 Update 4 current; long-tail dominant in fMRI labs given Chronos hardware integrationhttps://pstnet.com/products/e-prime/
Inquisit script1990s (Millisecond Software, Seattle)Millisecond SoftwareProprietary scripting language for stimulus presentation; .iqx scripts; Inquisit Web for online deploymentActive — Inquisit 6 line; dominant in implicit-association-test (IAT) and other reaction-time researchhttps://www.millisecond.com/
NBS Presentation (.exp / .sce / .pcl)1997 (Neurobehavioral Systems)Neurobehavioral Systems Inc. (Berkeley, CA)Proprietary stimulus-presentation language (PCL) with SDL scene-description sub-language; sub-millisecond timing claimsActive — long-tail dominant in EEG / MEG labs needing the tightest timing guaranteeshttps://www.neurobs.com/
PsyToolkit2010s (Gijsbert Stoet, Leeds Beckett)Open-source / StoetProprietary text-based scripting; web hosting; aimed at teaching and lightweight online studiesActivehttps://www.psytoolkit.org/

Notable threads

  • Qualtrics vs REDCap is the defining higher-ed split. Qualtrics dominates general university survey infrastructure (course evaluations, IRB-light social-science research, marketing studies) because its institutional licences are bundled with most U.S. R1 universities. REDCap dominates clinical-research university infrastructure because it is HIPAA-conformant, free to consortium members, and federated (each institution self-hosts). The 2026-era migration pressure runs in one direction: REDCap continues to absorb workloads Qualtrics cannot defensibly host (PHI, regulated trials), while Qualtrics’ AI features (response sentiment, theme extraction) push it deeper into corporate UX research. REDCap’s 16.x / 17.x lines in 2026 have introduced their own AI features (writing tone adjustment, free-text summarisation, automatic translation) — a deliberate response to Qualtrics’ XM iQ.

  • The XLSForm / ODK / KoBoToolbox stack is the global humanitarian standard. Almost every UN agency, MSF country office, and major international NGO collects structured field data with this stack. The reason is not technical brilliance but operational realism: XLSForm is authorable in Excel by non-developers, ODK Collect runs offline on cheap Android devices for weeks, and Central / KoBo handle eventual synchronisation. The 2026 KoboCollect Android-8 requirement is the biggest operational change in years — it forces hardware refresh cycles in low-resource deployments. SurveyCTO, Ona, and CommCare are all commercial extensions of the same XForms substrate.

  • jsPsych has become the de facto online psychology platform. The 8.x rewrite (2024+) modernised the timeline and plugin API, and v8.2.3 (Feb 2026) is the current production line. The combination jsPsych + Prolific (participant marketplace) + JATOS or Pavlovia (hosting) + OSF (pre-registration) is now the default reference architecture for online behavioural studies — replacing what was, a decade ago, a fragmented mess of MTurk + custom Flash / Java applets. PsychoPy retains the in-lab niche (eye-tracker, EEG, fMRI integration), but its PsychoJS export targets the same browser-based deployment as jsPsych.

  • The pre-registration / Open Science Framework drive is making experiment scripts public artefacts. A consequence of the post-2015 replication crisis is that experiment scripts (jsPsych code, .psyexp files, .osexp archives) are now expected to be deposited on OSF with a hash, alongside the pre-registration document. This pulls the format wars into the open: a .psyexp XML file or a jsPsych GitHub repo is now part of the published record, citable by DOI. The implication is conservative-by-default: PsychoPy and jsPsych benefit because their files are diff-able plain text, while E-Prime’s binary .ebs and Inquisit’s encrypted scripts feel increasingly archaic.

  • The CRF / EDC vendor lock-in problem and ODM-XML’s slow rescue. Historically, every commercial EDC vendor (Medidata Rave, Oracle Inform, Veeva CDB, OpenClinica) used a proprietary internal CRF representation. CDISC ODM-XML (1999, v2.0 in 2023) is the slow-burn answer: regulators (FDA, EMA, PMDA) require submission-side standards (SDTM, ADaM), which forces upstream tools to support ODM-XML or Define-XML for round-tripping. ODM v2.0’s optional JSON serialisation and FHIR Questionnaire’s growing footprint are the two trends slowly eroding the lock-in. The 2023 backwards-incompatibility break in ODM 2.0 is a once-in-a-decade event and most EDCs are still mid-migration through 2026.

  • The proliferation of JSON-based web survey schemas. SurveyJS, Typeform, Tally, Forms.app, Google Forms, and Microsoft Forms all settled on JSON as their wire format, but none agree on the shape — there is no surveyJSON analogue of OpenAPI. SurveyJS is the most influential open standard because its JSON is well-documented and it has React/Angular/Vue/vanilla runtimes. The pull toward JSON Schema (and by extension api-description) is real but resisted because survey-specific concerns (branching expressions, randomisation, piped text) outgrow JSON Schema’s vocabulary quickly. SurveyJS v3.0’s UI Preset Editor (Q1 2026) hints at the eventual answer: a sidecar “presentation/style schema” alongside the data schema, similar to how OpenAPI and JSON Schema separate.

  • Millisecond timing is the moat that keeps proprietary tools alive. The reason E-Prime, NBS Presentation, and Inquisit have not been swept away by jsPsych and PsychoPy is dedicated timing hardware (E-Prime’s Chronos response box, NBS’s purpose-built I/O cards). For visual psychophysics or EEG/fMRI synchronisation where 1 ms jitter matters, browser-based JS still has occasional limitations (refresh-rate negotiation, OS scheduling), and the proprietary stack maintains a defensible niche even though every other dimension (cost, sharing, reproducibility) favours the open tools.

Citations

Graph View