Skip to the guide
‹ Back to the app CivilKit Buckling · Guide

Worked examples and the Verified badge

A worked example is a fixture model, the inputs, the numbers you expect, and where those numbers come from. When every example reproduces its numbers, the module is Verified. The rules are CivilKit Studio's, unchanged.

The format#

tests/worked-examples.json (or the path in the manifest's tests) is a JSON list. Each example:

KeyWhat it is
idA short name, shown in the reports and in a failing install's status line.
inputThe inputs dict check receives, as the form would send it (blank numbers are "").
modelThe fixture: what getModel returns during this example, in the model shape, and the model every other capability works on by default. Optional for a module that never reads the model.
expected{"result": [{"label", "value", "unit"?}, …]}: the rows that must appear in what check returns.
sourceWhere the expected numbers come from: a hand calculation with its working, or a published example with its edition, example number and page. Shown under the example in every report.

The complete file of the module on the overview, with a hand calc:

[
  {
    "id": "lipped-c-150-60-40",
    "input": {"fy":450},
    "model": {
      "mats": [{"id":100,"ex":200000,"ey":200000,"vx":0.3,"vy":0.3,"g":76923.0769231}],
      "nodes": [
        {"x":60,"z":40,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":30,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":20,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":10,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":0,"free":[1,1,1,1],"stress":1},
        {"x":45,"z":0,"free":[1,1,1,1],"stress":1},
        {"x":30,"z":0,"free":[1,1,1,1],"stress":1},
        {"x":15,"z":0,"free":[1,1,1,1],"stress":1},
        {"x":0,"z":0,"free":[1,1,1,1],"stress":1},
        {"x":0,"z":37.5,"free":[1,1,1,1],"stress":1},
        {"x":0,"z":75,"free":[1,1,1,1],"stress":1},
        {"x":0,"z":112.5,"free":[1,1,1,1],"stress":1},
        {"x":0,"z":150,"free":[1,1,1,1],"stress":1},
        {"x":15,"z":150,"free":[1,1,1,1],"stress":1},
        {"x":30,"z":150,"free":[1,1,1,1],"stress":1},
        {"x":45,"z":150,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":150,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":140,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":130,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":120,"free":[1,1,1,1],"stress":1},
        {"x":60,"z":110,"free":[1,1,1,1],"stress":1}
      ],
      "elems": [
        {"i":0,"j":1,"t":1.5,"mat":100},
        {"i":1,"j":2,"t":1.5,"mat":100},
        {"i":2,"j":3,"t":1.5,"mat":100},
        {"i":3,"j":4,"t":1.5,"mat":100},
        {"i":4,"j":5,"t":1.5,"mat":100},
        {"i":5,"j":6,"t":1.5,"mat":100},
        {"i":6,"j":7,"t":1.5,"mat":100},
        {"i":7,"j":8,"t":1.5,"mat":100},
        {"i":8,"j":9,"t":1.5,"mat":100},
        {"i":9,"j":10,"t":1.5,"mat":100},
        {"i":10,"j":11,"t":1.5,"mat":100},
        {"i":11,"j":12,"t":1.5,"mat":100},
        {"i":12,"j":13,"t":1.5,"mat":100},
        {"i":13,"j":14,"t":1.5,"mat":100},
        {"i":14,"j":15,"t":1.5,"mat":100},
        {"i":15,"j":16,"t":1.5,"mat":100},
        {"i":16,"j":17,"t":1.5,"mat":100},
        {"i":17,"j":18,"t":1.5,"mat":100},
        {"i":18,"j":19,"t":1.5,"mat":100},
        {"i":19,"j":20,"t":1.5,"mat":100}
      ],
      "springs": [],
      "constraints": []
    },
    "expected": {"result":[{"label":"Py","value":236.25,"unit":"kN"}]},
    "source": "Hand calc: lipped C 150 x 60 x 40 mm (centreline), t = 1.5 mm, sharp corners. A = (150 + 2 x 60 + 2 x 40) x 1.5 = 525 mm2; Py = A fy = 525 x 450 = 236 250 N = 236.25 kN."
  }
]

How an example runs#

  • The module is loaded afresh for each example, so nothing carries over from the one before.
  • check(input) runs with the example's model behind every capability. Nothing is solved: hasResults() is False and getResults() is empty. A proposeModel call is recorded, never applied.
  • Anything that stops the run (a Python error, an undeclared capability, a model the engine refuses) becomes that example's only diff, and the other examples still run. For example, a fixture with a zero-thickness element fails with element 1 has thickness 0.
  • build_ui is not part of the test.

Matching rules#

Each expected row is looked up in the result by its label:

ExpectedPasses whenDiff when not
a numberthe result's value is a number within 0.1 % of it (an absolute floor of 1e-9 for expected values near zero)Py: expected 240, got 236.25
a stringthe value, as text, is exactly the sameGoverns: expected "local", got "distortional"
a unitthe result row's unit is exactly the same string (mm² is not mm2)A: expected unit mm2, got mm²
any rowa row with that label existsLcrl: missing from result

Rows the module returns beyond the expected ones are ignored, so an example can pin the two numbers you can prove and leave the rest. Leave out unit to skip the unit check for a row. Labels must match exactly, so keep the labels in a module stable once its examples are written.

0.1 % is tight on purpose. If your module rounds what it displays, round to enough figures: five significant figures is plenty; two decimal places on a value like 0.0123 is not.

The badge#

  • Verified: at least one example, and every example passes.
  • Community: anything else: no examples (or an empty list), a failing example, a run that stopped, or a file that is not a JSON list.

The app decides it when a module is installed, by running the examples in the sandbox in this browser, one at a time, each within the usual 20-second limit; the first failure ends it, and the status line names that example and its diff. The builder's Test and the command line run every example and report each.

"Checks nothing yet"#

An example whose expected.result is empty has nothing to compare, so it passes. By Studio's rule it still counts toward Verified. The builder flags it under the example, This example states no expected numbers, so it checks nothing yet., and its Use the current model as a fixture button always starts an example that way, for you to fill in. The command line does not warn, so look for empty expected lists when you review a module. A Verified badge resting on an empty example proves only that the module ran.

Sources: why expected values must not come from the engine#

Never copy the engine's output, or your module's, into expected. A test that compares the engine with itself always passes and proves nothing. The expected numbers must come from somewhere a checker can follow without running this app.

Good sources, roughly in order of strength:

  • A hand calculation you can show in full in source: the area and squash load of a sharp-cornered section (the centreline length times t), a DSM strength from given Pcr values, a property of a single plate.
  • A published design example: the edition, the example number, and its inputs. The DSM compression module uses the AISI Direct Strength Method Design Guide (CF06-1, 2006), Examples 8.1-4, 8.1-5 and 8.5-3.
  • A published reference model with its recorded output: CUFSM's own template run in its original code, or a finite strip curve shipped with a design guide. The Rounded C / Z module's examples are CUFSM's template, run in Octave, with the properties it recorded.

Printed precision. A guide prints two to four significant figures, and the tolerance is 0.1 %. Work the guide's own equations from its own printed inputs to more digits, put the working in your README, and state the printed value beside it, as the DSM compression README does (Pnl = 19.403 kip, which the guide prints as 19.4). Expected values are then the guide's equations, not your module's.

Units. Fixtures are in N, mm and MPa. Convert a published example once, state the factors in source (1 in = 25.4 mm, 1 ksi = 6.894757 MPa, 1 kip = 4.448222 kN), and convert the expected values the same way.

Buckling values. A minimum on a signature curve depends on the lengths it is solved at. A fixture that keeps the source's own lengths (in analysis.lengths) reproduces the source's minima; one with other lengths finds a slightly different minimum. The tutorial's second example does this.

Writing fixtures#

  • In the builder, Use the current model as a fixture copies the section on screen, its loads, fy and analysis settings into a new example, with the preview's inputs. Shape the section in the app first.
  • Include springs and constraints (empty lists are fine): a capability given a model without them stops with Cannot read properties of undefined (reading 'flat').
  • Element node indices are 0-based; the app's node 1 is 0.
  • Keep fixtures as small as the check allows: a fixture is part of the bundle, and every example re-solves it.
  • One example per behaviour worth proving: each branch of a code check (braced and unbraced, local and distortional governing) is a separate example.

CivilKit Buckling runs cufsm-rs, a Rust port of CUFSM, the finite strip software by Benjamin W. Schafer and co-workers (constrained FSM with Sándor Ádány), compiled to WebAssembly. An analysis tool, not a design certificate: check what you use.