From 8ff7bc9dc3a10590776bccd41bd81b00ea36f40b Mon Sep 17 00:00:00 2001 From: e4rohan Date: Thu, 22 Jan 2026 16:05:47 +0000 Subject: [PATCH 1/2] Add comprehensive documentation overhaul with best practices and patterns - Create BestPractices.md with opinionated design guidelines - Core principles (single responsibility, explicit dependencies) - Component design patterns for blocks, spaces, state - Naming conventions and coding standards - Common pitfalls and how to avoid them - Scaling considerations for small to large systems - Create ArchitecturalPatterns.md with reusable patterns - Pipeline, Fan-Out/Fan-In, State Machine patterns - Repository, Observer, Strategy, Adapter patterns - Pattern combinations and anti-patterns - Pattern selection guide - Create UsageGuidelines.md for when/how to use MSML - Ideal use cases and poor fits - Getting started workflow - Usage stages from ideation to maintenance - Integration strategies and success metrics - Update docs/index.md with organized documentation guide - Addresses issue #652 --- docs/ArchitecturalPatterns.md | 541 ++++++++++++++++++++++++++++++++++ docs/BestPractices.md | 433 +++++++++++++++++++++++++++ docs/UsageGuidelines.md | 440 +++++++++++++++++++++++++++ docs/index.md | 22 ++ 4 files changed, 1436 insertions(+) create mode 100644 docs/ArchitecturalPatterns.md create mode 100644 docs/BestPractices.md create mode 100644 docs/UsageGuidelines.md diff --git a/docs/ArchitecturalPatterns.md b/docs/ArchitecturalPatterns.md new file mode 100644 index 0000000..f47051d --- /dev/null +++ b/docs/ArchitecturalPatterns.md @@ -0,0 +1,541 @@ +# Architectural Patterns in MSML + +This guide presents common architectural patterns for structuring MSML specifications. These patterns help solve recurring design challenges and promote consistency. + +## Table of Contents + +- [Pipeline Pattern](#pipeline-pattern) +- [Fan-Out/Fan-In Pattern](#fan-outfan-in-pattern) +- [State Machine Pattern](#state-machine-pattern) +- [Repository Pattern](#repository-pattern) +- [Observer Pattern](#observer-pattern) +- [Strategy Pattern](#strategy-pattern) +- [Adapter Pattern](#adapter-pattern) + +## Pipeline Pattern + +### Intent + +Process data through a sequence of transformations, where each stage's output becomes the next stage's input. + +### When to Use + +- Sequential data processing +- Multi-step validation +- Graduated transformations +- Clear stage boundaries + +### Structure + +``` +Input → Stage1 → Stage2 → Stage3 → Output +``` + +### MSML Implementation + +```json +{ + "blocks": [ + { + "name": "ValidateInput", + "domain": ["RawTransaction"], + "codomain": ["ValidatedTransaction"] + }, + { + "name": "EnrichTransaction", + "domain": ["ValidatedTransaction"], + "codomain": ["EnrichedTransaction"] + }, + { + "name": "ExecuteTransaction", + "domain": ["EnrichedTransaction"], + "codomain": ["TransactionResult"] + } + ], + "wiring": [ + { + "name": "TransactionPipeline", + "components": [ + {"name": "ValidateInput"}, + {"name": "EnrichTransaction"}, + {"name": "ExecuteTransaction"} + ] + } + ] +} +``` + +### Benefits + +- Clear data flow +- Easy to test each stage +- Simple to add/remove stages +- Natural parallelization opportunities + +### Considerations + +- Error handling between stages +- Pipeline breakage on validation failure +- Performance of chained operations + +## Fan-Out/Fan-In Pattern + +### Intent + +Distribute work across multiple parallel operations, then combine results. + +### When to Use + +- Independent parallel computations +- Aggregating multiple data sources +- Load distribution +- Parallel validation checks + +### Structure + +``` + ┌→ Operation1 ─┐ +Input ────→┼→ Operation2 ─┼→ Combine → Output + └→ Operation3 ─┘ +``` + +### MSML Implementation + +```json +{ + "blocks": [ + { + "name": "CalculateMetric1", + "domain": ["SystemState"], + "codomain": ["Metric1"] + }, + { + "name": "CalculateMetric2", + "domain": ["SystemState"], + "codomain": ["Metric2"] + }, + { + "name": "CalculateMetric3", + "domain": ["SystemState"], + "codomain": ["Metric3"] + }, + { + "name": "AggregateMetrics", + "domain": ["Metric1", "Metric2", "Metric3"], + "codomain": ["OverallScore"] + } + ], + "wiring": [ + { + "name": "ParallelMetrics", + "components": [ + ["CalculateMetric1", "CalculateMetric2", "CalculateMetric3"], + ["AggregateMetrics"] + ] + } + ] +} +``` + +### Benefits + +- Parallel execution potential +- Modular metric calculation +- Easy to add new metrics +- Clear aggregation point + +### Considerations + +- Synchronization of parallel paths +- Handling partial failures +- Combining heterogeneous results + +## State Machine Pattern + +### Intent + +Model system behavior as explicit states with defined transitions. + +### When to Use + +- Clear lifecycle phases +- Status-dependent behavior +- Workflow management +- Approval processes + +### Structure + +``` +State A →[Event1]→ State B →[Event2]→ State C + ↑ ↓ + └──────────[Event3]────────────────────┘ +``` + +### MSML Implementation + +```json +{ + "state": [ + { + "name": "ProposalStatus", + "type": "enum", + "values": ["Pending", "Active", "Approved", "Rejected", "Executed"] + } + ], + "mechanisms": [ + { + "name": "TransitionToActive", + "domain": ["Proposal"], + "codomain": ["UpdatedProposal"], + "constraints": ["Proposal.status == 'Pending'"], + "updates": [{"state": "ProposalStatus", "to": "Active"}] + }, + { + "name": "ApproveProposal", + "domain": ["Proposal", "VoteResults"], + "codomain": ["UpdatedProposal"], + "constraints": [ + "Proposal.status == 'Active'", + "VoteResults.approved > VoteResults.threshold" + ], + "updates": [{"state": "ProposalStatus", "to": "Approved"}] + } + ] +} +``` + +### Benefits + +- Explicit state transitions +- Validation of valid transitions +- Clear business logic +- Audit trail of state changes + +### Considerations + +- State explosion with complex workflows +- Transition validation +- Concurrent state modifications + +## Repository Pattern + +### Intent + +Centralize data access and state management behind a well-defined interface. + +### When to Use + +- Multiple access patterns to same data +- Complex queries on state +- Abstraction over storage details +- Separation of data access from business logic + +### MSML Implementation + +```json +{ + "state": [ + { + "name": "UserRepository", + "type": "Dict[Address, User]", + "description": "Central user data store" + } + ], + "blocks": [ + { + "name": "GetUser", + "description": "Retrieve user by address", + "domain": ["Address", "UserRepository"], + "codomain": ["User"] + }, + { + "name": "QueryActiveUsers", + "description": "Find all active users", + "domain": ["UserRepository"], + "codomain": ["List[User]"] + }, + { + "name": "UpdateUser", + "description": "Update user data", + "domain": ["Address", "UserData", "UserRepository"], + "codomain": ["UpdatedRepository"] + } + ] +} +``` + +### Benefits + +- Centralized data access +- Consistent query interface +- Easier to optimize +- Clear data ownership + +### Considerations + +- Repository complexity growth +- Query performance +- State mutation coordination + +## Observer Pattern + +### Intent + +Notify interested components when state changes occur. + +### When to Use + +- Event-driven architectures +- Loose coupling between components +- Audit logging +- Reactive updates + +### MSML Implementation + +```json +{ + "boundary_actions": [ + { + "name": "BalanceChanged", + "description": "Emitted when balance updates", + "codomain": ["BalanceChangeEvent"] + } + ], + "mechanisms": [ + { + "name": "UpdateBalance", + "domain": ["Address", "NewBalance"], + "codomain": ["UpdatedBalance"], + "updates": [ + {"state": "Balances", "target": "Address"} + ], + "emits": ["BalanceChanged"] + } + ], + "policies": [ + { + "name": "OnBalanceChanged", + "description": "React to balance changes", + "domain": ["BalanceChangeEvent"], + "codomain": ["Action"] + } + ] +} +``` + +### Benefits + +- Decoupled components +- Easy to add observers +- Event history tracking +- Reactive behavior + +### Considerations + +- Event ordering +- Observer management +- Circular dependencies +- Event storm potential + +## Strategy Pattern + +### Intent + +Define a family of interchangeable algorithms/policies that can be selected at runtime. + +### When to Use + +- Multiple ways to perform an operation +- A/B testing different approaches +- User-configurable behavior +- Parameterized decision-making + +### MSML Implementation + +```json +{ + "policies": [ + { + "name": "FeePolicy", + "description": "Calculate transaction fee", + "domain": ["Transaction", "FeeParameters"], + "codomain": ["FeeAmount"], + "policy_options": [ + { + "name": "FlatFee", + "description": "Fixed fee per transaction", + "logic": "return fee_parameters.flat_rate" + }, + { + "name": "PercentageFee", + "description": "Percentage of transaction amount", + "logic": "return transaction.amount * fee_parameters.percentage" + }, + { + "name": "TieredFee", + "description": "Fee based on amount tiers", + "logic": "return calculate_tiered_fee(transaction.amount, fee_parameters.tiers)" + } + ] + } + ] +} +``` + +### Benefits + +- Algorithm flexibility +- Easy to add strategies +- Testable alternatives +- Clear strategy interface + +### Considerations + +- Strategy selection logic +- Parameter compatibility +- Performance differences +- Testing all strategies + +## Adapter Pattern + +### Intent + +Convert between incompatible interfaces or data formats. + +### When to Use + +- Integrating external systems +- Legacy compatibility +- Format transformations +- Protocol translation + +### MSML Implementation + +```json +{ + "blocks": [ + { + "name": "ExternalDataAdapter", + "description": "Convert external API format to internal format", + "domain": ["ExternalAPIResponse"], + "codomain": ["InternalDataFormat"] + }, + { + "name": "LegacySystemAdapter", + "description": "Adapt legacy state structure to new format", + "domain": ["LegacyState"], + "codomain": ["ModernState"] + } + ] +} +``` + +### Benefits + +- External system integration +- Format flexibility +- Backward compatibility +- Isolated conversions + +### Considerations + +- Conversion overhead +- Data loss in translation +- Versioning adapters +- Testing mappings + +## Combining Patterns + +### Example: Transaction Processing System + +Combines Pipeline, Strategy, and Observer patterns: + +```json +{ + "wiring": [ + { + "name": "TransactionProcessing", + "components": [ + "ValidateTransaction", // Pipeline step 1 + "ApplyFeePolicy", // Strategy pattern + "ExecuteTransaction", // Pipeline step 2 + "EmitTransactionEvent", // Observer pattern + "UpdateBalances" // Pipeline step 3 + ] + } + ] +} +``` + +### Example: Governance System + +Combines State Machine, Repository, and Fan-Out patterns: + +```json +{ + "wiring": [ + { + "name": "ProposalLifecycle", + "components": [ + "CreateProposal", // State Machine: → Pending + ["ValidateProposal", // Fan-out validation + "CheckEligibility", + "AssessRisks"], + "AggregateValidation", // Fan-in + "ActivateProposal", // State Machine: → Active + "StoreInRepository" // Repository pattern + ] + } + ] +} +``` + +## Anti-Patterns to Avoid + +### 1. The Big Ball of Mud + +**Problem**: No clear structure, everything depends on everything + +**Solution**: Apply separation of concerns, use clear patterns + +### 2. Premature Abstraction + +**Problem**: Over-engineering before understanding requirements + +**Solution**: Start simple, refactor when patterns emerge + +### 3. Pattern Obsession + +**Problem**: Forcing patterns where they don't fit + +**Solution**: Use patterns to solve problems, not for their own sake + +### 4. Tight Coupling + +**Problem**: Components too dependent on internal details + +**Solution**: Use interfaces (spaces), depend on abstractions + +### 5. God Object + +**Problem**: One component does everything + +**Solution**: Single responsibility principle, decompose + +## Pattern Selection Guide + +| Scenario | Recommended Pattern | +|----------|---------------------| +| Sequential processing | Pipeline | +| Parallel computations | Fan-Out/Fan-In | +| Lifecycle management | State Machine | +| Data access | Repository | +| Event notifications | Observer | +| Algorithm variants | Strategy | +| External integration | Adapter | + +## Further Reading + +- [Best Practices](BestPractices.md) - General design guidelines +- [Usage Guidelines](UsageGuidelines.md) - When to use MSML +- [Examples](CanonicalExamples.md) - Annotated examples diff --git a/docs/BestPractices.md b/docs/BestPractices.md new file mode 100644 index 0000000..8831666 --- /dev/null +++ b/docs/BestPractices.md @@ -0,0 +1,433 @@ +# MSML Best Practices + +This guide provides opinionated recommendations for building effective, maintainable mathematical specifications with MSML. + +## Table of Contents + +- [Core Principles](#core-principles) +- [Component Design](#component-design) +- [Naming Conventions](#naming-conventions) +- [Specification Organization](#specification-organization) +- [Common Pitfalls](#common-pitfalls) +- [Scaling Considerations](#scaling-considerations) + +## Core Principles + +### 1. Single Responsibility + +Each component (Block, Space, State variable) should have one clear purpose. + +**Good:** +- `CalculateReward` - Computes reward based on performance +- `UpdateBalance` - Updates user balance from transaction +- `ValidateTransaction` - Checks transaction validity + +**Bad:** +- `ProcessEverything` - Does multiple unrelated operations +- `UpdateAndValidate` - Mixes validation and state updates + +### 2. Explicit Over Implicit + +Make dependencies and data flows explicit through proper domain/codomain declarations. + +**Good:** +```json +{ + "name": "CalculateInterest", + "domain": ["Balance", "InterestRate"], + "codomain": ["InterestAmount"] +} +``` + +**Bad:** +```json +{ + "name": "CalculateInterest", + "domain": [], // Missing dependencies + "codomain": [] +} +``` + +### 3. Composability + +Design components to be composable and reusable across different contexts. + +**Good:** +- Small, focused blocks that can be combined +- Generic spaces that work with multiple blocks +- Reusable metrics across policies + +**Bad:** +- Large monolithic blocks that can't be decomposed +- Tightly coupled components +- Context-specific implementations that can't be reused + +## Component Design + +### Blocks + +**Blocks represent atomic operations in your system.** + +#### Design Guidelines + +1. **Focus**: One clear transformation or computation +2. **Size**: Aim for 3-7 inputs/outputs (cognitive load limit) +3. **Stateless**: Blocks should be pure functions when possible +4. **Testable**: Clear inputs/outputs make testing straightforward + +#### Good Block Design + +```json +{ + "name": "CalculateVotingPower", + "description": "Calculates voting power based on token balance and lock duration", + "domain": ["TokenBalance", "LockDuration"], + "codomain": ["VotingPower"], + "parameters_used": ["VotingPowerMultiplier"], + "constraints": ["TokenBalance > 0", "LockDuration >= MinLockDuration"] +} +``` + +#### Poor Block Design + +```json +{ + "name": "ProcessVote", + "description": "Does everything related to voting", + "domain": ["Everything"], // Too broad + "codomain": ["Results"], // Too vague + // Unclear what this block actually does +} +``` + +### Spaces + +**Spaces define the types and structure of data flowing between blocks.** + +#### Design Guidelines + +1. **Type Safety**: Clearly specify variable types +2. **Validation**: Include domain constraints +3. **Documentation**: Explain the meaning and units of variables +4. **Granularity**: One space per logical data group + +#### Good Space Design + +```json +{ + "name": "TokenBalance", + "schema": { + "amount": { + "type": "float", + "description": "Token amount in base units", + "constraints": "amount >= 0" + }, + "locked": { + "type": "boolean", + "description": "Whether tokens are locked" + } + } +} +``` + +### State Variables + +**State represents the persistent data in your system.** + +#### Design Guidelines + +1. **Minimal**: Only include truly persistent state +2. **Normalized**: Avoid redundant or derivable state +3. **Clear Ownership**: Each state variable should have clear update rules +4. **Versioned**: Consider versioning for evolving state structures + +#### State Organization + +```json +{ + "state": [ + { + "name": "UserBalances", + "type": "Dict[Address, Balance]", + "description": "Mapping of user addresses to token balances", + "updated_by": ["Transfer", "Mint", "Burn"] + }, + { + "name": "TotalSupply", + "type": "float", + "description": "Total token supply", + "updated_by": ["Mint", "Burn"] + } + ] +} +``` + +### Policies vs Mechanisms + +**Policies** = *What* to do (decision logic) +**Mechanisms** = *How* to update state (state transitions) + +#### Policy Guidelines + +- Focus on decision-making logic +- Return action parameters, not state changes +- Can be parameterized by user preferences +- Multiple policy options for different strategies + +#### Mechanism Guidelines + +- Focus on state update logic +- Deterministic given inputs +- Validate inputs before updating state +- Document update semantics clearly + +## Naming Conventions + +### General Rules + +1. **Descriptive**: Names should clearly indicate purpose +2. **Consistent**: Use consistent patterns across the spec +3. **Avoid Abbreviations**: Unless universally understood +4. **CamelCase**: For multi-word names + +### Component Naming Patterns + +| Component | Pattern | Example | +|-----------|---------|---------| +| Blocks | Verb + Noun | `CalculateReward`, `ValidateTransaction` | +| Spaces | Noun | `TokenBalance`, `UserProfile` | +| States | Noun (plural if collection) | `Balances`, `CurrentEpoch` | +| Parameters | Descriptive noun | `InterestRate`, `MaxSupply` | +| Policies | Verb + Context | `SelectValidator`, `DetermineReward` | +| Mechanisms | Update + Target | `UpdateBalance`, `TransferTokens` | + +### Variable Naming + +```python +# Good +user_balance +total_supply +interest_rate +transaction_amount + +# Bad +ub # Too short +x1 # Meaningless +tmp # Generic +data # Too vague +``` + +## Specification Organization + +### File Structure + +Organize complex specs into multiple files: + +``` +spec/ +├── main.json # Top-level spec with references +├── types/ +│ ├── tokens.json +│ └── users.json +├── state/ +│ ├── balances.json +│ └── governance.json +├── mechanisms/ +│ ├── transfer.json +│ └── minting.json +└── policies/ + ├── fee_policy.json + └── reward_policy.json +``` + +### Modular Design + +Break large systems into logical modules: + +1. **By Domain**: Group related functionality + - Token economics + - Governance + - Staking + +2. **By Layer**: Separate concerns + - Data layer (state, types) + - Logic layer (policies, mechanisms) + - Presentation layer (displays) + +3. **By Lifecycle**: Separate temporal concerns + - Initialization + - Regular operations + - Cleanup/finalization + +## Common Pitfalls + +### 1. Over-coupling + +**Problem**: Components depend on too many other components + +**Solution**: Use intermediate spaces to decouple + +```json +// Bad: Mechanism depends on multiple unrelated state variables +{ + "domain": ["UserBalance", "SystemConfig", "MarketPrice", "TimeOfDay"], + ... +} + +// Good: Introduce focused intermediate spaces +{ + "domain": ["TransactionContext"], // Aggregates what's needed + ... +} +``` + +### 2. God Blocks + +**Problem**: Single block does too much + +**Solution**: Decompose into smaller blocks + +```json +// Bad +{ + "name": "ProcessTransaction", + "description": "Validates, executes, logs, and notifies" +} + +// Good - Split into pipeline +[ + {"name": "ValidateTransaction"}, + {"name": "ExecuteTransaction"}, + {"name": "RecordTransaction"}, + {"name": "EmitNotification"} +] +``` + +### 3. Implicit Dependencies + +**Problem**: Hidden dependencies make reasoning difficult + +**Solution**: Make all dependencies explicit + +```json +// Bad +{ + "parameters_used": [] // Actually uses parameters internally +} + +// Good +{ + "parameters_used": ["InterestRate", "CompoundingFrequency"] +} +``` + +### 4. Leaky Abstractions + +**Problem**: Implementation details leak into specification + +**Solution**: Focus on *what*, not *how* + +```json +// Bad +{ + "description": "Uses binary search to find the optimal value" +} + +// Good +{ + "description": "Finds the optimal fee value given constraints" +} +``` + +### 5. Inconsistent Granularity + +**Problem**: Some blocks are atomic, others are coarse-grained + +**Solution**: Maintain consistent level of abstraction + +- Either stay high-level (conceptual) +- Or go detailed (implementation-ready) +- Don't mix both in same spec + +## Scaling Considerations + +### Small Systems (< 10 blocks) + +- Single file is fine +- Focus on clarity over optimization +- Inline documentation sufficient + +### Medium Systems (10-50 blocks) + +- Split into logical modules +- Use wiring diagrams to show composition +- Maintain glossary of terms +- Consider versioning strategy + +### Large Systems (50+ blocks) + +- Multi-file organization essential +- Formal documentation system (Obsidian vault) +- Clear ownership of components +- Automated validation in CI +- Version control for spec evolution +- Regular refactoring to reduce complexity + +### Handling Complexity + +When systems grow too complex: + +1. **Introduce Layers**: Separate high-level and detailed views +2. **Use Composition**: Build complex behaviors from simple blocks +3. **Abstract Patterns**: Extract common patterns into reusable components +4. **Hierarchical Organization**: Group related components +5. **Progressive Disclosure**: Show detail only when needed + +## Testing Your Specification + +### Validation Checklist + +- [ ] All blocks have clear domain/codomain +- [ ] All dependencies are explicit +- [ ] State update paths are documented +- [ ] Naming is consistent and descriptive +- [ ] No circular dependencies +- [ ] Constraints are specified +- [ ] Types are defined +- [ ] Parameters are documented + +### Review Questions + +1. Can a new contributor understand each component's purpose? +2. Are data flows clear without reading implementation? +3. Can components be tested independently? +4. Is the spec at consistent level of detail? +5. Are there obvious redundancies or inefficiencies? + +## Continuous Improvement + +### Refactoring Indicators + +Signs your spec needs refactoring: + +- Adding new features is difficult +- Many components change together +- Difficulty explaining data flows +- Lots of special cases +- Growing parameter lists +- Naming becomes unclear + +### Evolution Strategies + +1. **Backward Compatibility**: Version major changes +2. **Deprecation Path**: Mark old components before removal +3. **Migration Guides**: Document how to adapt to changes +4. **Changelog**: Track specification evolution +5. **Git Tags**: Mark stable versions + +## Additional Resources + +- [Architectural Patterns](ArchitecturalPatterns.md) - Common design patterns +- [Usage Guidelines](UsageGuidelines.md) - When and how to use MSML +- [Examples](CanonicalExamples.md) - Annotated examples +- [FAQ](FAQ.md) - Common questions and answers diff --git a/docs/UsageGuidelines.md b/docs/UsageGuidelines.md new file mode 100644 index 0000000..61cf7e7 --- /dev/null +++ b/docs/UsageGuidelines.md @@ -0,0 +1,440 @@ +# MSML Usage Guidelines + +This guide helps you understand when and how to use MSML effectively, and when alternative approaches might be more appropriate. + +## Table of Contents + +- [What is MSML For?](#what-is-msml-for) +- [When to Use MSML](#when-to-use-msml) +- [When NOT to Use MSML](#when-not-to-use-msml) +- [Getting Started](#getting-started) +- [Usage Stages](#usage-stages) +- [Integration Strategies](#integration-strategies) + +## What is MSML For? + +MSML (Mathematical Specification Mapping Library) is designed for: + +### 1. **Formal System Specification** + +Creating precise, unambiguous specifications of complex systems with: +- Multiple interacting components +- Complex state management +- Data dependencies +- Mathematical relationships + +### 2. **Communication and Documentation** + +Bridging the gap between: +- **Domain experts** who understand the system +- **Developers** who implement it +- **Stakeholders** who need to understand it +- **Auditors** who need to verify it + +### 3. **System Analysis and Validation** + +Enabling: +- Dependency analysis (what affects what) +- Impact assessment (change propagation) +- Consistency checking +- Coverage analysis + +### 4. **Automated Documentation** + +Generating: +- Human-readable reports +- Interactive visualizations +- Cross-referenced documentation +- Audit trails + +## When to Use MSML + +### Ideal Use Cases + +#### 1. **Token Economics and Protocol Design** + +MSML excels at specifying: +- Token supply mechanisms +- Reward distributions +- Fee structures +- Governance protocols +- Staking mechanisms + +**Why**: Complex interdependencies, multiple policies, state evolution over time + +**Example**: DeFi protocol with staking, rewards, and governance + +#### 2. **Complex Business Logic** + +Use MSML when you have: +- Multiple decision paths +- Configurable policies +- State machines with many transitions +- Regulatory requirements + +**Why**: Need for clarity, auditability, and documentation + +**Example**: Insurance claim processing, financial derivatives + +#### 3. **Multi-Stage Processes** + +MSML works well for: +- Approval workflows +- Transaction processing pipelines +- Data transformation chains +- Sequential validations + +**Why**: Clear stage boundaries, dependency tracking + +**Example**: KYC/AML verification pipeline + +#### 4. **Simulation and Model-Based Testing** + +Use MSML when: +- Testing different scenarios +- Comparing policy options +- Stress testing systems +- Validating economic models + +**Why**: Parameterized alternatives, reproducible scenarios + +**Example**: Testing different fee structures before deployment + +#### 5. **Collaborative Design** + +MSML helps when: +- Multiple teams working on one system +- Stakeholders need visibility +- Documentation must stay current +- Changes need review + +**Why**: Single source of truth, automated docs, version control + +**Example**: Cross-functional protocol development + +### Good Indicators for MSML + +- ✅ System has 5+ distinct operations +- ✅ Multiple components interact +- ✅ State evolution is complex +- ✅ Dependencies are non-trivial +- ✅ Documentation is critical +- ✅ Multiple stakeholders involved +- ✅ Regular spec changes expected +- ✅ Need to compare alternatives + +## When NOT to Use MSML + +### Poor Fits + +#### 1. **Simple, Straightforward Logic** + +Don't use MSML for: +- Simple CRUD operations +- Straightforward calculations +- Trivial state updates +- One-off scripts + +**Why**: Overhead exceeds benefits + +**Alternative**: Just write the code directly + +#### 2. **Pure Implementation Details** + +Avoid MSML for: +- Database schema design +- API endpoint definitions +- UI component structure +- Performance optimizations + +**Why**: MSML focuses on logic, not implementation + +**Alternative**: Standard architecture documentation + +#### 3. **Rapidly Changing Requirements** + +MSML may not fit when: +- Requirements are highly uncertain +- Frequent pivots expected +- Prototyping phase +- Exploring problem space + +**Why**: Spec maintenance overhead + +**Alternative**: Start with prototypes, spec later once stable + +#### 4. **Very Small Systems** + +Skip MSML for: +- Single-function systems +- Minimal state +- Few interactions +- No complex logic + +**Why**: Documentation overhead isn't justified + +**Alternative**: Code comments and basic docs + +#### 5. **Performance-Critical Code** + +Don't spec with MSML: +- Low-level optimizations +- Hardware-specific code +- Real-time constraints +- Micro-optimizations + +**Why**: Implementation details matter more than abstract logic + +**Alternative**: Profiling-driven development + +### Warning Signs Against MSML + +- ❌ "We just need to build a simple form" +- ❌ "Requirements change daily" +- ❌ "We're just exploring ideas" +- ❌ "It's mostly UI work" +- ❌ "Performance is the only concern" +- ❌ "We have 2 components total" + +## Getting Started + +### 1. Start Small + +Begin with a focused subsystem: + +``` +✓ Start: "Token transfer mechanism" +✗ Avoid: "Entire DeFi ecosystem" +``` + +### 2. Identify Core Components + +Focus on: +- **State**: What persists? +- **Blocks**: What operations exist? +- **Spaces**: What data flows between blocks? +- **Wirings**: How do operations compose? + +### 3. Iterate and Expand + +1. **Spec core logic** (20% effort) +2. **Validate with stakeholders** +3. **Add details** (30% effort) +4. **Generate documentation** +5. **Refine based on feedback** (50% effort) +6. **Expand to adjacent systems** + +### 4. Integrate with Development + +Don't treat spec as separate from code: +- Sync spec changes with code changes +- Use spec for code review +- Generate tests from spec +- Link spec to implementation + +## Usage Stages + +### Stage 1: Ideation and Requirements + +**Activities:** +- Capture system components as you discover them +- Use MSML types for domain concepts +- Create informal block descriptions +- Build basic state model + +**Artifacts:** +- High-level component list +- Initial state variables +- Basic type definitions + +**Tools:** +- Markdown with wiki-links +- MSML JSON (minimal) +- Diagrams (informal) + +### Stage 2: Design and Specification + +**Activities:** +- Formalize block definitions +- Specify spaces and data flows +- Define policies and mechanisms +- Document constraints and assumptions + +**Artifacts:** +- Complete MSML JSON specification +- Domain/codomain definitions +- Parameter specifications +- Constraint documentation + +**Tools:** +- MSML JSON editor +- Validation tools +- Report generation + +### Stage 3: Implementation + +**Activities:** +- Use spec to guide coding +- Link implementation to spec blocks +- Validate implementations match spec +- Keep spec and code in sync + +**Artifacts:** +- Implementation code +- Spec-to-code mappings +- Unit tests based on spec +- Integration tests + +**Tools:** +- MSML function implementations +- Code generation (if applicable) +- Test generation + +### Stage 4: Validation and Testing + +**Activities:** +- Compare spec alternatives (policy options) +- Run simulations +- Stress test edge cases +- Validate economic properties + +**Artifacts:** +- Simulation results +- Test coverage reports +- Performance metrics +- Economic analysis + +**Tools:** +- MSML wiring execution +- Simulation frameworks +- Analysis scripts + +### Stage 5: Maintenance and Evolution + +**Activities:** +- Update spec as system evolves +- Track spec changes in version control +- Generate updated documentation +- Review impact of changes + +**Artifacts:** +- Spec change history +- Updated documentation +- Migration guides +- Deprecation notices + +**Tools:** +- Git for versioning +- Automated doc generation +- Change impact analysis + +## Integration Strategies + +### Strategy 1: Spec-First Development + +**Process:** +1. Write MSML spec +2. Review with stakeholders +3. Implement from spec +4. Validate against spec + +**Best for:** +- Well-understood domains +- Critical systems +- Regulated environments +- Team collaboration + +### Strategy 2: Spec-Alongside Development + +**Process:** +1. Prototype implementation +2. Extract spec from prototype +3. Formalize spec +4. Refine both in parallel + +**Best for:** +- Exploratory projects +- Novel domains +- Rapid iterations +- Learning environments + +### Strategy 3: Spec-After Documentation + +**Process:** +1. Build system +2. Reverse-engineer spec +3. Use for documentation +4. Maintain going forward + +**Best for:** +- Legacy systems +- Undocumented systems +- Onboarding new teams +- Knowledge transfer + +### Strategy 4: Hybrid Approach + +**Process:** +1. Spec critical paths upfront +2. Prototype non-critical parts +3. Formalize as you go +4. Consolidate at milestones + +**Best for:** +- Mixed complexity systems +- Phased projects +- Risk-managed development +- Resource-constrained teams + +## Measuring Success + +### Specification Quality Metrics + +- **Completeness**: Are all components specified? +- **Consistency**: Do definitions align? +- **Clarity**: Can stakeholders understand it? +- **Maintainability**: Easy to update? +- **Traceability**: Clear links to implementation? + +### Usage Effectiveness Indicators + +**Positive signs:** +- Faster onboarding for new team members +- Fewer misunderstandings in reviews +- Easier impact analysis for changes +- Stakeholder confidence in design +- Reduced debugging time + +**Warning signs:** +- Spec constantly out of sync with code +- Team avoids updating spec +- Stakeholders don't use generated docs +- More time on spec than implementation +- Spec feels like busywork + +## Recommendations by Project Type + +| Project Type | MSML Usage | Focus Areas | +|--------------|------------|-------------| +| **DeFi Protocol** | High | Token mechanics, governance, security | +| **NFT Platform** | Medium | Minting, transfer, royalty logic | +| **DAO Governance** | High | Proposal lifecycle, voting, execution | +| **Supply Chain** | Medium | State tracking, transitions, auditing | +| **Gaming Economy** | High | Resource flows, player actions, balance | +| **Simple dApp** | Low | Consider simpler docs | +| **Microservice** | Low | Use API specs instead | +| **UI Application** | Very Low | Focus on component docs | + +## Getting Help + +- **Documentation**: Start with [Getting Started](GettingStarted.md) +- **Patterns**: See [Architectural Patterns](ArchitecturalPatterns.md) +- **Best Practices**: Read [Best Practices](BestPractices.md) +- **Examples**: Study [Canonical Examples](CanonicalExamples.md) +- **Community**: Engage with MSML users and contributors + +## Conclusion + +MSML is a powerful tool for specifying complex systems, but it's not right for every project. Use this guide to determine if MSML fits your needs, and if so, how to integrate it effectively into your development process. + +**Remember**: The goal is better systems, not better specs. Use MSML where it helps achieve that goal. diff --git a/docs/index.md b/docs/index.md index e8d0012..4059ed2 100644 --- a/docs/index.md +++ b/docs/index.md @@ -16,6 +16,28 @@ One good example is the [wiring report](https://github.com/BlockScience/Predator To install the library, simply pip install by running "pip install math_spec_mapping" +For development setup, see the [main repository](https://github.com/BlockScience/MSML) for instructions using [uv](https://docs.astral.sh/uv/). + +## Documentation Guide + +### Getting Started +- **[Getting Started](GettingStarted.md)** - Installation and first steps +- **[Usage Guidelines](UsageGuidelines.md)** - When and how to use MSML +- **[Best Practices](BestPractices.md)** - Opinionated design recommendations +- **[Architectural Patterns](ArchitecturalPatterns.md)** - Common design patterns + +### Reference +- **[GDS Fundamentals](GDS%20Fundamentals.md)** - Generalized Dynamical Systems basics +- **[JSON Specification](JSON-Specification/schema.md)** - Complete JSON schema reference +- **[Technical Documentation](TechnicalDocumentation.md)** - API and technical details +- **[MSML Enhancements](MSML%20Enhancements.md)** - Advanced features + +### Examples and Tools +- **[Canonical Examples](CanonicalExamples.md)** - Annotated example specifications +- **[Reports](Reports.md)** - Report generation guide +- **[cadCAD Builder](cadCADBuilder.md)** - Integration with cadCAD +- **[FAQ](FAQ.md)** - Frequently asked questions + ## Why MSML? Writing mathematical specifications can be a difficult process, especially when variable names are changed or new mechanisms are introduced. MSML seeks to streamline the process with automations as well as enhance the abilities of static math specs to deliver deeper insights. Because it is automated, one can write specifications at different levels of details or for different purposes. From 64de796b8485120d34a474f5d5712179a47749a5 Mon Sep 17 00:00:00 2001 From: e4rohan Date: Thu, 22 Jan 2026 16:34:47 +0000 Subject: [PATCH 2/2] removed non validated --- docs/ArchitecturalPatterns.md | 541 ---------------------------------- docs/BestPractices.md | 433 --------------------------- docs/UsageGuidelines.md | 440 --------------------------- docs/index.md | 3 - 4 files changed, 1417 deletions(-) delete mode 100644 docs/ArchitecturalPatterns.md delete mode 100644 docs/BestPractices.md delete mode 100644 docs/UsageGuidelines.md diff --git a/docs/ArchitecturalPatterns.md b/docs/ArchitecturalPatterns.md deleted file mode 100644 index f47051d..0000000 --- a/docs/ArchitecturalPatterns.md +++ /dev/null @@ -1,541 +0,0 @@ -# Architectural Patterns in MSML - -This guide presents common architectural patterns for structuring MSML specifications. These patterns help solve recurring design challenges and promote consistency. - -## Table of Contents - -- [Pipeline Pattern](#pipeline-pattern) -- [Fan-Out/Fan-In Pattern](#fan-outfan-in-pattern) -- [State Machine Pattern](#state-machine-pattern) -- [Repository Pattern](#repository-pattern) -- [Observer Pattern](#observer-pattern) -- [Strategy Pattern](#strategy-pattern) -- [Adapter Pattern](#adapter-pattern) - -## Pipeline Pattern - -### Intent - -Process data through a sequence of transformations, where each stage's output becomes the next stage's input. - -### When to Use - -- Sequential data processing -- Multi-step validation -- Graduated transformations -- Clear stage boundaries - -### Structure - -``` -Input → Stage1 → Stage2 → Stage3 → Output -``` - -### MSML Implementation - -```json -{ - "blocks": [ - { - "name": "ValidateInput", - "domain": ["RawTransaction"], - "codomain": ["ValidatedTransaction"] - }, - { - "name": "EnrichTransaction", - "domain": ["ValidatedTransaction"], - "codomain": ["EnrichedTransaction"] - }, - { - "name": "ExecuteTransaction", - "domain": ["EnrichedTransaction"], - "codomain": ["TransactionResult"] - } - ], - "wiring": [ - { - "name": "TransactionPipeline", - "components": [ - {"name": "ValidateInput"}, - {"name": "EnrichTransaction"}, - {"name": "ExecuteTransaction"} - ] - } - ] -} -``` - -### Benefits - -- Clear data flow -- Easy to test each stage -- Simple to add/remove stages -- Natural parallelization opportunities - -### Considerations - -- Error handling between stages -- Pipeline breakage on validation failure -- Performance of chained operations - -## Fan-Out/Fan-In Pattern - -### Intent - -Distribute work across multiple parallel operations, then combine results. - -### When to Use - -- Independent parallel computations -- Aggregating multiple data sources -- Load distribution -- Parallel validation checks - -### Structure - -``` - ┌→ Operation1 ─┐ -Input ────→┼→ Operation2 ─┼→ Combine → Output - └→ Operation3 ─┘ -``` - -### MSML Implementation - -```json -{ - "blocks": [ - { - "name": "CalculateMetric1", - "domain": ["SystemState"], - "codomain": ["Metric1"] - }, - { - "name": "CalculateMetric2", - "domain": ["SystemState"], - "codomain": ["Metric2"] - }, - { - "name": "CalculateMetric3", - "domain": ["SystemState"], - "codomain": ["Metric3"] - }, - { - "name": "AggregateMetrics", - "domain": ["Metric1", "Metric2", "Metric3"], - "codomain": ["OverallScore"] - } - ], - "wiring": [ - { - "name": "ParallelMetrics", - "components": [ - ["CalculateMetric1", "CalculateMetric2", "CalculateMetric3"], - ["AggregateMetrics"] - ] - } - ] -} -``` - -### Benefits - -- Parallel execution potential -- Modular metric calculation -- Easy to add new metrics -- Clear aggregation point - -### Considerations - -- Synchronization of parallel paths -- Handling partial failures -- Combining heterogeneous results - -## State Machine Pattern - -### Intent - -Model system behavior as explicit states with defined transitions. - -### When to Use - -- Clear lifecycle phases -- Status-dependent behavior -- Workflow management -- Approval processes - -### Structure - -``` -State A →[Event1]→ State B →[Event2]→ State C - ↑ ↓ - └──────────[Event3]────────────────────┘ -``` - -### MSML Implementation - -```json -{ - "state": [ - { - "name": "ProposalStatus", - "type": "enum", - "values": ["Pending", "Active", "Approved", "Rejected", "Executed"] - } - ], - "mechanisms": [ - { - "name": "TransitionToActive", - "domain": ["Proposal"], - "codomain": ["UpdatedProposal"], - "constraints": ["Proposal.status == 'Pending'"], - "updates": [{"state": "ProposalStatus", "to": "Active"}] - }, - { - "name": "ApproveProposal", - "domain": ["Proposal", "VoteResults"], - "codomain": ["UpdatedProposal"], - "constraints": [ - "Proposal.status == 'Active'", - "VoteResults.approved > VoteResults.threshold" - ], - "updates": [{"state": "ProposalStatus", "to": "Approved"}] - } - ] -} -``` - -### Benefits - -- Explicit state transitions -- Validation of valid transitions -- Clear business logic -- Audit trail of state changes - -### Considerations - -- State explosion with complex workflows -- Transition validation -- Concurrent state modifications - -## Repository Pattern - -### Intent - -Centralize data access and state management behind a well-defined interface. - -### When to Use - -- Multiple access patterns to same data -- Complex queries on state -- Abstraction over storage details -- Separation of data access from business logic - -### MSML Implementation - -```json -{ - "state": [ - { - "name": "UserRepository", - "type": "Dict[Address, User]", - "description": "Central user data store" - } - ], - "blocks": [ - { - "name": "GetUser", - "description": "Retrieve user by address", - "domain": ["Address", "UserRepository"], - "codomain": ["User"] - }, - { - "name": "QueryActiveUsers", - "description": "Find all active users", - "domain": ["UserRepository"], - "codomain": ["List[User]"] - }, - { - "name": "UpdateUser", - "description": "Update user data", - "domain": ["Address", "UserData", "UserRepository"], - "codomain": ["UpdatedRepository"] - } - ] -} -``` - -### Benefits - -- Centralized data access -- Consistent query interface -- Easier to optimize -- Clear data ownership - -### Considerations - -- Repository complexity growth -- Query performance -- State mutation coordination - -## Observer Pattern - -### Intent - -Notify interested components when state changes occur. - -### When to Use - -- Event-driven architectures -- Loose coupling between components -- Audit logging -- Reactive updates - -### MSML Implementation - -```json -{ - "boundary_actions": [ - { - "name": "BalanceChanged", - "description": "Emitted when balance updates", - "codomain": ["BalanceChangeEvent"] - } - ], - "mechanisms": [ - { - "name": "UpdateBalance", - "domain": ["Address", "NewBalance"], - "codomain": ["UpdatedBalance"], - "updates": [ - {"state": "Balances", "target": "Address"} - ], - "emits": ["BalanceChanged"] - } - ], - "policies": [ - { - "name": "OnBalanceChanged", - "description": "React to balance changes", - "domain": ["BalanceChangeEvent"], - "codomain": ["Action"] - } - ] -} -``` - -### Benefits - -- Decoupled components -- Easy to add observers -- Event history tracking -- Reactive behavior - -### Considerations - -- Event ordering -- Observer management -- Circular dependencies -- Event storm potential - -## Strategy Pattern - -### Intent - -Define a family of interchangeable algorithms/policies that can be selected at runtime. - -### When to Use - -- Multiple ways to perform an operation -- A/B testing different approaches -- User-configurable behavior -- Parameterized decision-making - -### MSML Implementation - -```json -{ - "policies": [ - { - "name": "FeePolicy", - "description": "Calculate transaction fee", - "domain": ["Transaction", "FeeParameters"], - "codomain": ["FeeAmount"], - "policy_options": [ - { - "name": "FlatFee", - "description": "Fixed fee per transaction", - "logic": "return fee_parameters.flat_rate" - }, - { - "name": "PercentageFee", - "description": "Percentage of transaction amount", - "logic": "return transaction.amount * fee_parameters.percentage" - }, - { - "name": "TieredFee", - "description": "Fee based on amount tiers", - "logic": "return calculate_tiered_fee(transaction.amount, fee_parameters.tiers)" - } - ] - } - ] -} -``` - -### Benefits - -- Algorithm flexibility -- Easy to add strategies -- Testable alternatives -- Clear strategy interface - -### Considerations - -- Strategy selection logic -- Parameter compatibility -- Performance differences -- Testing all strategies - -## Adapter Pattern - -### Intent - -Convert between incompatible interfaces or data formats. - -### When to Use - -- Integrating external systems -- Legacy compatibility -- Format transformations -- Protocol translation - -### MSML Implementation - -```json -{ - "blocks": [ - { - "name": "ExternalDataAdapter", - "description": "Convert external API format to internal format", - "domain": ["ExternalAPIResponse"], - "codomain": ["InternalDataFormat"] - }, - { - "name": "LegacySystemAdapter", - "description": "Adapt legacy state structure to new format", - "domain": ["LegacyState"], - "codomain": ["ModernState"] - } - ] -} -``` - -### Benefits - -- External system integration -- Format flexibility -- Backward compatibility -- Isolated conversions - -### Considerations - -- Conversion overhead -- Data loss in translation -- Versioning adapters -- Testing mappings - -## Combining Patterns - -### Example: Transaction Processing System - -Combines Pipeline, Strategy, and Observer patterns: - -```json -{ - "wiring": [ - { - "name": "TransactionProcessing", - "components": [ - "ValidateTransaction", // Pipeline step 1 - "ApplyFeePolicy", // Strategy pattern - "ExecuteTransaction", // Pipeline step 2 - "EmitTransactionEvent", // Observer pattern - "UpdateBalances" // Pipeline step 3 - ] - } - ] -} -``` - -### Example: Governance System - -Combines State Machine, Repository, and Fan-Out patterns: - -```json -{ - "wiring": [ - { - "name": "ProposalLifecycle", - "components": [ - "CreateProposal", // State Machine: → Pending - ["ValidateProposal", // Fan-out validation - "CheckEligibility", - "AssessRisks"], - "AggregateValidation", // Fan-in - "ActivateProposal", // State Machine: → Active - "StoreInRepository" // Repository pattern - ] - } - ] -} -``` - -## Anti-Patterns to Avoid - -### 1. The Big Ball of Mud - -**Problem**: No clear structure, everything depends on everything - -**Solution**: Apply separation of concerns, use clear patterns - -### 2. Premature Abstraction - -**Problem**: Over-engineering before understanding requirements - -**Solution**: Start simple, refactor when patterns emerge - -### 3. Pattern Obsession - -**Problem**: Forcing patterns where they don't fit - -**Solution**: Use patterns to solve problems, not for their own sake - -### 4. Tight Coupling - -**Problem**: Components too dependent on internal details - -**Solution**: Use interfaces (spaces), depend on abstractions - -### 5. God Object - -**Problem**: One component does everything - -**Solution**: Single responsibility principle, decompose - -## Pattern Selection Guide - -| Scenario | Recommended Pattern | -|----------|---------------------| -| Sequential processing | Pipeline | -| Parallel computations | Fan-Out/Fan-In | -| Lifecycle management | State Machine | -| Data access | Repository | -| Event notifications | Observer | -| Algorithm variants | Strategy | -| External integration | Adapter | - -## Further Reading - -- [Best Practices](BestPractices.md) - General design guidelines -- [Usage Guidelines](UsageGuidelines.md) - When to use MSML -- [Examples](CanonicalExamples.md) - Annotated examples diff --git a/docs/BestPractices.md b/docs/BestPractices.md deleted file mode 100644 index 8831666..0000000 --- a/docs/BestPractices.md +++ /dev/null @@ -1,433 +0,0 @@ -# MSML Best Practices - -This guide provides opinionated recommendations for building effective, maintainable mathematical specifications with MSML. - -## Table of Contents - -- [Core Principles](#core-principles) -- [Component Design](#component-design) -- [Naming Conventions](#naming-conventions) -- [Specification Organization](#specification-organization) -- [Common Pitfalls](#common-pitfalls) -- [Scaling Considerations](#scaling-considerations) - -## Core Principles - -### 1. Single Responsibility - -Each component (Block, Space, State variable) should have one clear purpose. - -**Good:** -- `CalculateReward` - Computes reward based on performance -- `UpdateBalance` - Updates user balance from transaction -- `ValidateTransaction` - Checks transaction validity - -**Bad:** -- `ProcessEverything` - Does multiple unrelated operations -- `UpdateAndValidate` - Mixes validation and state updates - -### 2. Explicit Over Implicit - -Make dependencies and data flows explicit through proper domain/codomain declarations. - -**Good:** -```json -{ - "name": "CalculateInterest", - "domain": ["Balance", "InterestRate"], - "codomain": ["InterestAmount"] -} -``` - -**Bad:** -```json -{ - "name": "CalculateInterest", - "domain": [], // Missing dependencies - "codomain": [] -} -``` - -### 3. Composability - -Design components to be composable and reusable across different contexts. - -**Good:** -- Small, focused blocks that can be combined -- Generic spaces that work with multiple blocks -- Reusable metrics across policies - -**Bad:** -- Large monolithic blocks that can't be decomposed -- Tightly coupled components -- Context-specific implementations that can't be reused - -## Component Design - -### Blocks - -**Blocks represent atomic operations in your system.** - -#### Design Guidelines - -1. **Focus**: One clear transformation or computation -2. **Size**: Aim for 3-7 inputs/outputs (cognitive load limit) -3. **Stateless**: Blocks should be pure functions when possible -4. **Testable**: Clear inputs/outputs make testing straightforward - -#### Good Block Design - -```json -{ - "name": "CalculateVotingPower", - "description": "Calculates voting power based on token balance and lock duration", - "domain": ["TokenBalance", "LockDuration"], - "codomain": ["VotingPower"], - "parameters_used": ["VotingPowerMultiplier"], - "constraints": ["TokenBalance > 0", "LockDuration >= MinLockDuration"] -} -``` - -#### Poor Block Design - -```json -{ - "name": "ProcessVote", - "description": "Does everything related to voting", - "domain": ["Everything"], // Too broad - "codomain": ["Results"], // Too vague - // Unclear what this block actually does -} -``` - -### Spaces - -**Spaces define the types and structure of data flowing between blocks.** - -#### Design Guidelines - -1. **Type Safety**: Clearly specify variable types -2. **Validation**: Include domain constraints -3. **Documentation**: Explain the meaning and units of variables -4. **Granularity**: One space per logical data group - -#### Good Space Design - -```json -{ - "name": "TokenBalance", - "schema": { - "amount": { - "type": "float", - "description": "Token amount in base units", - "constraints": "amount >= 0" - }, - "locked": { - "type": "boolean", - "description": "Whether tokens are locked" - } - } -} -``` - -### State Variables - -**State represents the persistent data in your system.** - -#### Design Guidelines - -1. **Minimal**: Only include truly persistent state -2. **Normalized**: Avoid redundant or derivable state -3. **Clear Ownership**: Each state variable should have clear update rules -4. **Versioned**: Consider versioning for evolving state structures - -#### State Organization - -```json -{ - "state": [ - { - "name": "UserBalances", - "type": "Dict[Address, Balance]", - "description": "Mapping of user addresses to token balances", - "updated_by": ["Transfer", "Mint", "Burn"] - }, - { - "name": "TotalSupply", - "type": "float", - "description": "Total token supply", - "updated_by": ["Mint", "Burn"] - } - ] -} -``` - -### Policies vs Mechanisms - -**Policies** = *What* to do (decision logic) -**Mechanisms** = *How* to update state (state transitions) - -#### Policy Guidelines - -- Focus on decision-making logic -- Return action parameters, not state changes -- Can be parameterized by user preferences -- Multiple policy options for different strategies - -#### Mechanism Guidelines - -- Focus on state update logic -- Deterministic given inputs -- Validate inputs before updating state -- Document update semantics clearly - -## Naming Conventions - -### General Rules - -1. **Descriptive**: Names should clearly indicate purpose -2. **Consistent**: Use consistent patterns across the spec -3. **Avoid Abbreviations**: Unless universally understood -4. **CamelCase**: For multi-word names - -### Component Naming Patterns - -| Component | Pattern | Example | -|-----------|---------|---------| -| Blocks | Verb + Noun | `CalculateReward`, `ValidateTransaction` | -| Spaces | Noun | `TokenBalance`, `UserProfile` | -| States | Noun (plural if collection) | `Balances`, `CurrentEpoch` | -| Parameters | Descriptive noun | `InterestRate`, `MaxSupply` | -| Policies | Verb + Context | `SelectValidator`, `DetermineReward` | -| Mechanisms | Update + Target | `UpdateBalance`, `TransferTokens` | - -### Variable Naming - -```python -# Good -user_balance -total_supply -interest_rate -transaction_amount - -# Bad -ub # Too short -x1 # Meaningless -tmp # Generic -data # Too vague -``` - -## Specification Organization - -### File Structure - -Organize complex specs into multiple files: - -``` -spec/ -├── main.json # Top-level spec with references -├── types/ -│ ├── tokens.json -│ └── users.json -├── state/ -│ ├── balances.json -│ └── governance.json -├── mechanisms/ -│ ├── transfer.json -│ └── minting.json -└── policies/ - ├── fee_policy.json - └── reward_policy.json -``` - -### Modular Design - -Break large systems into logical modules: - -1. **By Domain**: Group related functionality - - Token economics - - Governance - - Staking - -2. **By Layer**: Separate concerns - - Data layer (state, types) - - Logic layer (policies, mechanisms) - - Presentation layer (displays) - -3. **By Lifecycle**: Separate temporal concerns - - Initialization - - Regular operations - - Cleanup/finalization - -## Common Pitfalls - -### 1. Over-coupling - -**Problem**: Components depend on too many other components - -**Solution**: Use intermediate spaces to decouple - -```json -// Bad: Mechanism depends on multiple unrelated state variables -{ - "domain": ["UserBalance", "SystemConfig", "MarketPrice", "TimeOfDay"], - ... -} - -// Good: Introduce focused intermediate spaces -{ - "domain": ["TransactionContext"], // Aggregates what's needed - ... -} -``` - -### 2. God Blocks - -**Problem**: Single block does too much - -**Solution**: Decompose into smaller blocks - -```json -// Bad -{ - "name": "ProcessTransaction", - "description": "Validates, executes, logs, and notifies" -} - -// Good - Split into pipeline -[ - {"name": "ValidateTransaction"}, - {"name": "ExecuteTransaction"}, - {"name": "RecordTransaction"}, - {"name": "EmitNotification"} -] -``` - -### 3. Implicit Dependencies - -**Problem**: Hidden dependencies make reasoning difficult - -**Solution**: Make all dependencies explicit - -```json -// Bad -{ - "parameters_used": [] // Actually uses parameters internally -} - -// Good -{ - "parameters_used": ["InterestRate", "CompoundingFrequency"] -} -``` - -### 4. Leaky Abstractions - -**Problem**: Implementation details leak into specification - -**Solution**: Focus on *what*, not *how* - -```json -// Bad -{ - "description": "Uses binary search to find the optimal value" -} - -// Good -{ - "description": "Finds the optimal fee value given constraints" -} -``` - -### 5. Inconsistent Granularity - -**Problem**: Some blocks are atomic, others are coarse-grained - -**Solution**: Maintain consistent level of abstraction - -- Either stay high-level (conceptual) -- Or go detailed (implementation-ready) -- Don't mix both in same spec - -## Scaling Considerations - -### Small Systems (< 10 blocks) - -- Single file is fine -- Focus on clarity over optimization -- Inline documentation sufficient - -### Medium Systems (10-50 blocks) - -- Split into logical modules -- Use wiring diagrams to show composition -- Maintain glossary of terms -- Consider versioning strategy - -### Large Systems (50+ blocks) - -- Multi-file organization essential -- Formal documentation system (Obsidian vault) -- Clear ownership of components -- Automated validation in CI -- Version control for spec evolution -- Regular refactoring to reduce complexity - -### Handling Complexity - -When systems grow too complex: - -1. **Introduce Layers**: Separate high-level and detailed views -2. **Use Composition**: Build complex behaviors from simple blocks -3. **Abstract Patterns**: Extract common patterns into reusable components -4. **Hierarchical Organization**: Group related components -5. **Progressive Disclosure**: Show detail only when needed - -## Testing Your Specification - -### Validation Checklist - -- [ ] All blocks have clear domain/codomain -- [ ] All dependencies are explicit -- [ ] State update paths are documented -- [ ] Naming is consistent and descriptive -- [ ] No circular dependencies -- [ ] Constraints are specified -- [ ] Types are defined -- [ ] Parameters are documented - -### Review Questions - -1. Can a new contributor understand each component's purpose? -2. Are data flows clear without reading implementation? -3. Can components be tested independently? -4. Is the spec at consistent level of detail? -5. Are there obvious redundancies or inefficiencies? - -## Continuous Improvement - -### Refactoring Indicators - -Signs your spec needs refactoring: - -- Adding new features is difficult -- Many components change together -- Difficulty explaining data flows -- Lots of special cases -- Growing parameter lists -- Naming becomes unclear - -### Evolution Strategies - -1. **Backward Compatibility**: Version major changes -2. **Deprecation Path**: Mark old components before removal -3. **Migration Guides**: Document how to adapt to changes -4. **Changelog**: Track specification evolution -5. **Git Tags**: Mark stable versions - -## Additional Resources - -- [Architectural Patterns](ArchitecturalPatterns.md) - Common design patterns -- [Usage Guidelines](UsageGuidelines.md) - When and how to use MSML -- [Examples](CanonicalExamples.md) - Annotated examples -- [FAQ](FAQ.md) - Common questions and answers diff --git a/docs/UsageGuidelines.md b/docs/UsageGuidelines.md deleted file mode 100644 index 61cf7e7..0000000 --- a/docs/UsageGuidelines.md +++ /dev/null @@ -1,440 +0,0 @@ -# MSML Usage Guidelines - -This guide helps you understand when and how to use MSML effectively, and when alternative approaches might be more appropriate. - -## Table of Contents - -- [What is MSML For?](#what-is-msml-for) -- [When to Use MSML](#when-to-use-msml) -- [When NOT to Use MSML](#when-not-to-use-msml) -- [Getting Started](#getting-started) -- [Usage Stages](#usage-stages) -- [Integration Strategies](#integration-strategies) - -## What is MSML For? - -MSML (Mathematical Specification Mapping Library) is designed for: - -### 1. **Formal System Specification** - -Creating precise, unambiguous specifications of complex systems with: -- Multiple interacting components -- Complex state management -- Data dependencies -- Mathematical relationships - -### 2. **Communication and Documentation** - -Bridging the gap between: -- **Domain experts** who understand the system -- **Developers** who implement it -- **Stakeholders** who need to understand it -- **Auditors** who need to verify it - -### 3. **System Analysis and Validation** - -Enabling: -- Dependency analysis (what affects what) -- Impact assessment (change propagation) -- Consistency checking -- Coverage analysis - -### 4. **Automated Documentation** - -Generating: -- Human-readable reports -- Interactive visualizations -- Cross-referenced documentation -- Audit trails - -## When to Use MSML - -### Ideal Use Cases - -#### 1. **Token Economics and Protocol Design** - -MSML excels at specifying: -- Token supply mechanisms -- Reward distributions -- Fee structures -- Governance protocols -- Staking mechanisms - -**Why**: Complex interdependencies, multiple policies, state evolution over time - -**Example**: DeFi protocol with staking, rewards, and governance - -#### 2. **Complex Business Logic** - -Use MSML when you have: -- Multiple decision paths -- Configurable policies -- State machines with many transitions -- Regulatory requirements - -**Why**: Need for clarity, auditability, and documentation - -**Example**: Insurance claim processing, financial derivatives - -#### 3. **Multi-Stage Processes** - -MSML works well for: -- Approval workflows -- Transaction processing pipelines -- Data transformation chains -- Sequential validations - -**Why**: Clear stage boundaries, dependency tracking - -**Example**: KYC/AML verification pipeline - -#### 4. **Simulation and Model-Based Testing** - -Use MSML when: -- Testing different scenarios -- Comparing policy options -- Stress testing systems -- Validating economic models - -**Why**: Parameterized alternatives, reproducible scenarios - -**Example**: Testing different fee structures before deployment - -#### 5. **Collaborative Design** - -MSML helps when: -- Multiple teams working on one system -- Stakeholders need visibility -- Documentation must stay current -- Changes need review - -**Why**: Single source of truth, automated docs, version control - -**Example**: Cross-functional protocol development - -### Good Indicators for MSML - -- ✅ System has 5+ distinct operations -- ✅ Multiple components interact -- ✅ State evolution is complex -- ✅ Dependencies are non-trivial -- ✅ Documentation is critical -- ✅ Multiple stakeholders involved -- ✅ Regular spec changes expected -- ✅ Need to compare alternatives - -## When NOT to Use MSML - -### Poor Fits - -#### 1. **Simple, Straightforward Logic** - -Don't use MSML for: -- Simple CRUD operations -- Straightforward calculations -- Trivial state updates -- One-off scripts - -**Why**: Overhead exceeds benefits - -**Alternative**: Just write the code directly - -#### 2. **Pure Implementation Details** - -Avoid MSML for: -- Database schema design -- API endpoint definitions -- UI component structure -- Performance optimizations - -**Why**: MSML focuses on logic, not implementation - -**Alternative**: Standard architecture documentation - -#### 3. **Rapidly Changing Requirements** - -MSML may not fit when: -- Requirements are highly uncertain -- Frequent pivots expected -- Prototyping phase -- Exploring problem space - -**Why**: Spec maintenance overhead - -**Alternative**: Start with prototypes, spec later once stable - -#### 4. **Very Small Systems** - -Skip MSML for: -- Single-function systems -- Minimal state -- Few interactions -- No complex logic - -**Why**: Documentation overhead isn't justified - -**Alternative**: Code comments and basic docs - -#### 5. **Performance-Critical Code** - -Don't spec with MSML: -- Low-level optimizations -- Hardware-specific code -- Real-time constraints -- Micro-optimizations - -**Why**: Implementation details matter more than abstract logic - -**Alternative**: Profiling-driven development - -### Warning Signs Against MSML - -- ❌ "We just need to build a simple form" -- ❌ "Requirements change daily" -- ❌ "We're just exploring ideas" -- ❌ "It's mostly UI work" -- ❌ "Performance is the only concern" -- ❌ "We have 2 components total" - -## Getting Started - -### 1. Start Small - -Begin with a focused subsystem: - -``` -✓ Start: "Token transfer mechanism" -✗ Avoid: "Entire DeFi ecosystem" -``` - -### 2. Identify Core Components - -Focus on: -- **State**: What persists? -- **Blocks**: What operations exist? -- **Spaces**: What data flows between blocks? -- **Wirings**: How do operations compose? - -### 3. Iterate and Expand - -1. **Spec core logic** (20% effort) -2. **Validate with stakeholders** -3. **Add details** (30% effort) -4. **Generate documentation** -5. **Refine based on feedback** (50% effort) -6. **Expand to adjacent systems** - -### 4. Integrate with Development - -Don't treat spec as separate from code: -- Sync spec changes with code changes -- Use spec for code review -- Generate tests from spec -- Link spec to implementation - -## Usage Stages - -### Stage 1: Ideation and Requirements - -**Activities:** -- Capture system components as you discover them -- Use MSML types for domain concepts -- Create informal block descriptions -- Build basic state model - -**Artifacts:** -- High-level component list -- Initial state variables -- Basic type definitions - -**Tools:** -- Markdown with wiki-links -- MSML JSON (minimal) -- Diagrams (informal) - -### Stage 2: Design and Specification - -**Activities:** -- Formalize block definitions -- Specify spaces and data flows -- Define policies and mechanisms -- Document constraints and assumptions - -**Artifacts:** -- Complete MSML JSON specification -- Domain/codomain definitions -- Parameter specifications -- Constraint documentation - -**Tools:** -- MSML JSON editor -- Validation tools -- Report generation - -### Stage 3: Implementation - -**Activities:** -- Use spec to guide coding -- Link implementation to spec blocks -- Validate implementations match spec -- Keep spec and code in sync - -**Artifacts:** -- Implementation code -- Spec-to-code mappings -- Unit tests based on spec -- Integration tests - -**Tools:** -- MSML function implementations -- Code generation (if applicable) -- Test generation - -### Stage 4: Validation and Testing - -**Activities:** -- Compare spec alternatives (policy options) -- Run simulations -- Stress test edge cases -- Validate economic properties - -**Artifacts:** -- Simulation results -- Test coverage reports -- Performance metrics -- Economic analysis - -**Tools:** -- MSML wiring execution -- Simulation frameworks -- Analysis scripts - -### Stage 5: Maintenance and Evolution - -**Activities:** -- Update spec as system evolves -- Track spec changes in version control -- Generate updated documentation -- Review impact of changes - -**Artifacts:** -- Spec change history -- Updated documentation -- Migration guides -- Deprecation notices - -**Tools:** -- Git for versioning -- Automated doc generation -- Change impact analysis - -## Integration Strategies - -### Strategy 1: Spec-First Development - -**Process:** -1. Write MSML spec -2. Review with stakeholders -3. Implement from spec -4. Validate against spec - -**Best for:** -- Well-understood domains -- Critical systems -- Regulated environments -- Team collaboration - -### Strategy 2: Spec-Alongside Development - -**Process:** -1. Prototype implementation -2. Extract spec from prototype -3. Formalize spec -4. Refine both in parallel - -**Best for:** -- Exploratory projects -- Novel domains -- Rapid iterations -- Learning environments - -### Strategy 3: Spec-After Documentation - -**Process:** -1. Build system -2. Reverse-engineer spec -3. Use for documentation -4. Maintain going forward - -**Best for:** -- Legacy systems -- Undocumented systems -- Onboarding new teams -- Knowledge transfer - -### Strategy 4: Hybrid Approach - -**Process:** -1. Spec critical paths upfront -2. Prototype non-critical parts -3. Formalize as you go -4. Consolidate at milestones - -**Best for:** -- Mixed complexity systems -- Phased projects -- Risk-managed development -- Resource-constrained teams - -## Measuring Success - -### Specification Quality Metrics - -- **Completeness**: Are all components specified? -- **Consistency**: Do definitions align? -- **Clarity**: Can stakeholders understand it? -- **Maintainability**: Easy to update? -- **Traceability**: Clear links to implementation? - -### Usage Effectiveness Indicators - -**Positive signs:** -- Faster onboarding for new team members -- Fewer misunderstandings in reviews -- Easier impact analysis for changes -- Stakeholder confidence in design -- Reduced debugging time - -**Warning signs:** -- Spec constantly out of sync with code -- Team avoids updating spec -- Stakeholders don't use generated docs -- More time on spec than implementation -- Spec feels like busywork - -## Recommendations by Project Type - -| Project Type | MSML Usage | Focus Areas | -|--------------|------------|-------------| -| **DeFi Protocol** | High | Token mechanics, governance, security | -| **NFT Platform** | Medium | Minting, transfer, royalty logic | -| **DAO Governance** | High | Proposal lifecycle, voting, execution | -| **Supply Chain** | Medium | State tracking, transitions, auditing | -| **Gaming Economy** | High | Resource flows, player actions, balance | -| **Simple dApp** | Low | Consider simpler docs | -| **Microservice** | Low | Use API specs instead | -| **UI Application** | Very Low | Focus on component docs | - -## Getting Help - -- **Documentation**: Start with [Getting Started](GettingStarted.md) -- **Patterns**: See [Architectural Patterns](ArchitecturalPatterns.md) -- **Best Practices**: Read [Best Practices](BestPractices.md) -- **Examples**: Study [Canonical Examples](CanonicalExamples.md) -- **Community**: Engage with MSML users and contributors - -## Conclusion - -MSML is a powerful tool for specifying complex systems, but it's not right for every project. Use this guide to determine if MSML fits your needs, and if so, how to integrate it effectively into your development process. - -**Remember**: The goal is better systems, not better specs. Use MSML where it helps achieve that goal. diff --git a/docs/index.md b/docs/index.md index 4059ed2..422ebc0 100644 --- a/docs/index.md +++ b/docs/index.md @@ -22,9 +22,6 @@ For development setup, see the [main repository](https://github.com/BlockScience ### Getting Started - **[Getting Started](GettingStarted.md)** - Installation and first steps -- **[Usage Guidelines](UsageGuidelines.md)** - When and how to use MSML -- **[Best Practices](BestPractices.md)** - Opinionated design recommendations -- **[Architectural Patterns](ArchitecturalPatterns.md)** - Common design patterns ### Reference - **[GDS Fundamentals](GDS%20Fundamentals.md)** - Generalized Dynamical Systems basics