Discounted Action Policy Workbench

How to use Discounted Action Policy Workbench

Solve the fixed-policy reward equations, compare all one-step action alternatives and apply only strict policy improvements until an exact certificate or the stated iteration cap.

1. Use the example as the schema; replace its labels and entered model assumptions.
2. Solve the fixed-policy reward equations, compare all one-step action alternatives and apply only strict policy improvements until an exact certificate or the stated iteration cap.
3. Calculate, inspect the complete witnesses and limitations, then copy or download the JSON report.

Good to know

• 1–6 distinct states, each with 1–3 distinct actions and a probability row matching the state order. Probabilities are exact decimal strings from 0 to 1 summing exactly to 1. Immediate rewards/costs have magnitude at most 1 billion; discount is 0–0.999999. All decimal strings have at most six places.
• policy is one zero-based action index per state. mode is evaluate or optimize; objective maximize or minimize. Optional iterationBudget is a plain JSON integer 1–100. A stop reports the evaluated current policy and unknown optimality; only zero Bellman gaps certify optimum. Model is stationary, fully observed, infinite horizon and discounted, without learned probabilities, real control advice or average-reward/finite-horizon claims.
• Local calculation with no upload, input persistence or outside lookup. Model JSON is limited to 30,000 characters and 10,000 values. Decimal quantities are strings with up to six places; count fields use plain JSON integers. Exact fractions are capped at 8,192 bits. The complete JSON output is limited to 1,000,000 characters; approximate numbers preserve their stored values. Unsupported states fail explicitly.