CLI User Guide¶
The pyrmute CLI provides command-line tools for validating data, migrating between versions, comparing schemas, and managing your model configurations. This guide covers installation, configuration, and all available commands.
Installation¶
Install pyrmute with CLI support:
Verify installation:
Quick Start¶
1. Initialize a Project¶
Create a new pyrmute project with example configuration:
This creates:
- models.py - Example model definitions
- pyproject.toml or pyrmute.toml - Configuration file
2. View Registered Models¶
See all models and versions:
Output:
3. Validate Data¶
Validate a JSON file against a schema version:
data.json
Configuration¶
Configuration Files¶
pyrmute looks for configuration in this order:
- Explicit config file (via
--configflag) pyproject.tomlwith[tool.pyrmute]sectionpyrmute.tomlin current directory- Auto-location of
__pyrmute_manager__in Python files
Basic Configuration¶
pyproject.toml:
pyrmute.toml:
This tells pyrmute to import the manager attribute from the models module.
Custom Manager Names¶
Use a specific attribute name:
This imports custom_manager from the models module.
Multiple Managers¶
Configure multiple managers for different purposes:
Use with the --manager flag:
Factory Functions¶
Use factory functions with initialization arguments:
[tool.pyrmute]
manager = "models:create_manager"
init_args = ["production"]
[tool.pyrmute.init_kwargs]
debug = false
cache_enabled = true
In models.py:
def create_manager(
env: str, debug: bool = False, cache_enabled: bool = True
) -> ModelManager:
"""Factory function for creating manager."""
mgr = ModelManager()
# Do stuff with arguments
return mgr
__pyrmute_manager__ = create_manager
Auto-Discovery¶
If no configuration file exists, pyrmute searches for:
models.pywith__pyrmute_manager__in current directory*_models.pyfiles with__pyrmute_manager__*/models.pyfiles with__pyrmute_manager__
Commands¶
pyrmute init¶
Initialize a new pyrmute project.
Basic usage:
Options:
--project-dir PATH- Directory to initialize (default: current)--pyproject- Use pyproject.toml instead of pyrmute.toml--multiple- Configure multiple managers
Examples:
Create single manager project:
Create with pyproject.toml:
Create multi-manager project:
Create in specific directory:
pyrmute info¶
Show information about registered models.
Basic usage:
Arguments:
MANAGER- Manager name (default: "default")
Options:
--config PATH- Path to config file
Examples:
Show default manager:
Show specific manager:
Use custom config:
Output:
pyrmute managers¶
List all available managers from configuration.
Basic usage:
Options:
--config PATH- Path to config file
Output:
┏━━━━━━━━━┳━━━━━━━━━━━━━━━━┓
┃ Name ┃ Module ┃
┡━━━━━━━━━╇━━━━━━━━━━━━━━━━┩
│ default │ models │
│ api_v1 │ api.v1.models │
│ api_v2 │ api.v2.models │
└─────────┴────────────────┘
Use with: pyrmute validate --manager <name> ...
pyrmute validate¶
Validate JSON data against a schema version.
Basic usage:
Required options:
-d, --data PATH- Path to JSON data file-s, --schema NAME- Schema name-v, --version VERSION- Schema version
Optional flags:
-m, --manager NAME- Manager name (default: "default")-c, --config PATH- Path to config file-s, --style STYLE- TypeScript output style: "interface", "type", "zod" (default: "interface", TypeScript only)
Examples:
Validate user data:
Validate with specific manager:
Use custom config:
Success output:
Failure output:
✗ Validation failed
Validation errors:
• email: Field required
• age: Input should be a valid integer
pyrmute migrate¶
Migrate data from one schema version to another.
Basic usage:
Required options:
-d, --data PATH- Path to JSON data file-s, --schema NAME- Schema name-f, --from VERSION- Source version-t, --to VERSION- Target version
Optional flags:
-o, --output PATH- Output file (default: stdout)-m, --manager NAME- Manager name (default: "default")-c, --config PATH- Path to config file
Examples:
Migrate to stdout:
Migrate to file:
Multi-version migration:
Success output (to file):
Success output (to stdout):
Failure output:
pyrmute diff¶
Show differences between schema versions.
Basic usage:
Required options:
-s, --schema NAME- Schema name-f, --from VERSION- Source version-t, --to VERSION- Target version
Optional flags:
--format FORMAT- Output format: "markdown" or "json" (default: "markdown")-m, --manager NAME- Manager name (default: "default")-c, --config PATH- Path to config file
Examples:
Show diff in markdown:
Show diff in JSON:
Compare with specific manager:
Markdown output:
# User: 1.0.0 → 2.0.0
## Added Fields
- `age: int | None` (optional)
- `phone: str` (required)
## Removed Fields
- `username`
## Modified Fields
- `email` - now optional
## Breaking Changes
- ⚠️ New required field 'phone' will fail for existing data without defaults
- ⚠️ Field 'email' changed from optional to required
- ⚠️ Removed fields 'username' will be lost during migration
JSON output:
{
"model_name": "User",
"from_version": "1.0.0",
"to_version": "2.0.0",
"added_fields": ["age", "phone"],
"removed_fields": ["username"],
"modified_fields": {
"email": {
"required_changed": {
"from": true,
"to": false
}
}
},
"added_field_info": {
"age": {
"type": "int | None",
"required": false,
"default": null
},
"phone": {
"type": "str",
"required": true,
"default": null
}
},
"unchanged_fields": ["name"]
}
pyrmute export¶
Export schemas to various formats.
Basic usage:
Required options:
-f, --format FORMAT- Export format-o, --output PATH- Output directory
Supported formats:
json-schema- JSON Schema formatavro- Apache Avro schemasprotobuf- Protocol Buffer schemastypescript- TypeScript interfaces and Zod schemas
Optional flags:
-m, --manager NAME- Manager name (default: "default")-c, --config PATH- Path to config file--organization STYLE- Directory organization (TypeScript only): "flat", "major_version", "model"--barrel-exports/--no-barrel-exports- Generate index.ts files (TypeScript only, default: enabled)
Examples:
Export JSON schemas:
Export Avro schemas:
Export ProtoBuf schemas:
Export TypeScript schemas (flat organization):
Export TypeScript with major_version organization (recommended):
Export TypeScript organized by model:
Export TypeScript without barrel exports:
Export TypeScript type aliases:
Export TypeScript Zod schemas with runtime validation:
Combine style with organization:
Success output:
Directory structure (flat):
Directory structure (major_version):
types/
├── v1/
│ ├── User.v1.0.0.ts
│ ├── User.v1.1.0.ts
│ ├── Order.v1.0.0.ts
│ ├── Product.v1.0.0.ts
│ └── index.ts
├── v2/
│ ├── User.v2.0.0.ts
│ ├── Order.v2.0.0.ts
│ └── index.ts
└── index.ts
Directory structure (model):
types/
├── User/
│ ├── 1.0.0.ts
│ ├── 1.1.0.ts
│ ├── 2.0.0.ts
│ └── index.ts
├── Order/
│ ├── 1.0.0.ts
│ ├── 2.0.0.ts
│ └── index.ts
├── Product/
│ ├── 1.0.0.ts
│ └── index.ts
└── index.ts
Note: The --style, --organization, and --barrel-exports flags only
apply to TypeScript exports. They are ignored for other formats.
Common Workflows¶
Development Workflow¶
1. Create new model version:
# models.py
@manager.model("User", "2.0.0")
class UserV2(BaseModel):
name: str
email: str
age: int | None = None # New field
@manager.migration("User", "1.0.0", "2.0.0")
def add_age_field(data):
return {**data, "age": None}
2. Check what changed:
3. Test migration:
4. Validate migrated data:
pyrmute migrate -d test_data.json -s User -f 1.0.0 -t 2.0.0 -o migrated.json
pyrmute validate -d migrated.json -s User -v 2.0.0
CI/CD Integration¶
Validate all test data:
#!/bin/bash
for file in test_data/*.json; do
pyrmute validate -d "$file" -s User -v 2.0.0 || exit 1
done
Export schemas for documentation:
# In CI pipeline
pyrmute export -f json-schema -o docs/schemas
pyrmute export -f typescript -o frontend/types --style interface
pyrmute export -f typescript -o frontend/schemas --style zod
Check for breaking changes:
# Compare versions and fail if breaking changes detected
pyrmute diff -s User -f 1.0.0 -t 2.0.0 --format json > diff.json
# Parse diff.json and check for breaking changes
python scripts/check_breaking_changes.py diff.json
Data Migration Pipeline¶
Migrate multiple files:
#!/bin/bash
for file in data/v1/*.json; do
filename=$(basename "$file")
pyrmute migrate \
-d "$file" \
-s User \
-f 1.0.0 \
-t 2.0.0 \
-o "data/v2/$filename"
done
Batch validation:
#!/bin/bash
# Validate all migrated files
for file in data/v2/*.json; do
echo "Validating $file..."
pyrmute validate -d "$file" -s User -v 2.0.0
done
Multi-Environment Setup¶
Development environment:
# dev.toml
[pyrmute]
manager = "models:create_manager"
init_args = ["development"]
[pyrmute.init_kwargs]
debug = true
Production environment:
# prod.toml
[pyrmute]
manager = "models:create_manager"
init_args = ["production"]
[pyrmute.init_kwargs]
debug = false
Usage:
# Development
pyrmute validate -d data.json -s User -v 2.0.0 --config dev.toml
# Production
pyrmute validate -d data.json -s User -v 2.0.0 --config prod.toml
Error Handling¶
Configuration Errors¶
Missing configuration:
Error: No pyrmute configuration found. Create a pyproject.toml or
pyrmute.toml, or define __pyrmute_manager__ in models.py
Solution: Run pyrmute init or create a configuration file.
Invalid module path:
Solution: Ensure the module path is correct and the file exists.
Manager not found:
Solution: Check manager name spelling or run pyrmute managers to list available managers.
Validation Errors¶
Schema not found:
Solution: Check available versions with pyrmute info.
Invalid JSON:
Error: Invalid JSON in data.json: Expecting property name enclosed in
double quotes: line 3 column 5 (char 25)
Solution: Validate JSON syntax with a JSON linter.
Validation failed:
✗ Validation failed
Validation errors:
• email: Field required
• age: Input should be a valid integer
Solution: Fix data to match schema requirements.
Migration Errors¶
No migration path:
Solution: Define migration functions for intermediate versions or create a direct migration.
Migration validation failed:
Solution: Update migration function to provide required fields.
Best Practices¶
1. Use Configuration Files¶
Always use configuration files rather than relying on auto-discovery:
2. Validate Before Migrating¶
Always validate data before attempting migration:
# Check current version is valid
pyrmute validate -d data.json -s User -v 1.0.0
# Then migrate
pyrmute migrate -d data.json -s User -f 1.0.0 -t 2.0.0 -o migrated.json
# Validate migrated data
pyrmute validate -d migrated.json -s User -v 2.0.0
3. Review Diffs Before Deploying¶
Check schema differences before deploying new versions:
Pay attention to breaking changes warnings.
4. Use Meaningful Manager Names¶
For multi-manager setups, use descriptive names:
[tool.pyrmute.managers]
public_api = "api.public.models"
internal_api = "api.internal.models"
admin_api = "api.admin.models"
5. Export Schemas Regularly¶
Export schemas for documentation and external tools:
# Export during build process
pyrmute export -f json-schema -o docs/api/schemas
pyrmute export -f typescript -o frontend/src/types --style interface
pyrmute export -f typescript -o frontend/src/schemas --style zod
6. Version Control Configuration¶
Always commit configuration files:
7. Test Migrations Locally¶
Test migrations with sample data before production:
# Create test data
cat > test_user.json << EOF
{
"name": "Test User",
"email": "test@example.com"
}
EOF
# Test migration
pyrmute migrate -d test_user.json -s User -f 1.0.0 -t 2.0.0
Scripting and Automation¶
Bash Scripts¶
Validate all files in directory:
#!/bin/bash
set -e
SCHEMA="User"
VERSION="2.0.0"
for file in data/*.json; do
echo "Validating $file..."
pyrmute validate -d "$file" -s "$SCHEMA" -v "$VERSION"
done
echo "All files validated successfully!"
Migrate with backup:
#!/bin/bash
set -e
INPUT="$1"
SCHEMA="$2"
FROM="$3"
TO="$4"
# Create backup
cp "$INPUT" "${INPUT}.backup"
# Migrate
pyrmute migrate \
-d "$INPUT" \
-s "$SCHEMA" \
-f "$FROM" \
-t "$TO" \
-o "$INPUT.migrated"
# Validate migrated data
if pyrmute validate -d "$INPUT.migrated" -s "$SCHEMA" -v "$TO"; then
mv "$INPUT.migrated" "$INPUT"
echo "Migration successful!"
else
echo "Migration validation failed, restoring backup"
mv "${INPUT}.backup" "$INPUT"
exit 1
fi
Python Scripts¶
Batch processing with error handling:
#!/usr/bin/env python3
import subprocess
import sys
from pathlib import Path
def migrate_file(input_file: Path, schema: str, from_ver: str, to_ver: str) -> bool:
"""Migrate a single file."""
output_file = input_file.with_suffix('.migrated.json')
try:
result = subprocess.run(
[
'pyrmute', 'migrate',
'-d', str(input_file),
'-s', schema,
'-f', from_ver,
'-t', to_ver,
'-o', str(output_file)
],
check=True,
capture_output=True,
text=True
)
print(f"✓ Migrated {input_file}")
return True
except subprocess.CalledProcessError as e:
print(f"✗ Failed to migrate {input_file}")
print(e.stderr)
return False
def main() -> None:
data_dir = Path('data/v1')
schema = 'User'
from_version = '1.0.0'
to_version = '2.0.0'
files = list(data_dir.glob('*.json'))
success_count = 0
for file in files:
if migrate_file(file, schema, from_version, to_version):
success_count += 1
print(f"\nMigrated {success_count}/{len(files)} files successfully")
sys.exit(0 if success_count == len(files) else 1)
if __name__ == '__main__':
main()
Makefile Integration¶
.PHONY: validate migrate export clean
# Validate all test data
validate:
@echo "Validating test data..."
@for file in tests/data/*.json; do \
pyrmute validate -d $$file -s User -v 2.0.0 || exit 1; \
done
# Migrate data from v1 to v2
migrate:
@echo "Migrating data..."
@mkdir -p data/v2
@for file in data/v1/*.json; do \
filename=$$(basename $$file); \
pyrmute migrate \
-d $$file \
-s User \
-f 1.0.0 \
-t 2.0.0 \
-o data/v2/$$filename; \
done
# Export schemas for documentation
export:
@echo "Exporting schemas..."
@mkdir -p docs/schemas
@pyrmute export -f json-schema -o docs/schemas
# Clean generated files
clean:
@rm -rf data/v2 docs/schemas
Troubleshooting¶
Command Not Found¶
Error:
Solution:
# Ensure pyrmute is installed
pip install pyrmute[cli]
# Check installation
pip show pyrmute
# Add to PATH if needed
export PATH="$HOME/.local/bin:$PATH"
Import Errors¶
Error:
Solution:
- Ensure you're in the correct directory
- Check that
models.pyexists - Verify PYTHONPATH includes current directory
Module Cache Issues¶
If you're getting stale data in tests:
# Clear Python cache
find . -type d -name __pycache__ -exec rm -rf {} +
find . -type f -name "*.pyc" -delete
Permission Errors¶
Error:
Solution:
# Check file permissions
ls -l output.json
# Fix permissions
chmod 644 output.json
# Or use different output location
pyrmute migrate -d data.json -s User -f 1.0.0 -t 2.0.0 -o /tmp/output.json
Next Steps¶
Now that you understand the CLI:
User Guides:
- Registering Models - Define your models
- Writing Migrations - Write migration functions
- Schema Generation - Schema generation functionality
Advanced Topics:
- Custom Generators - Use custom schema generators
- Migration Hooks - Add hooks to migrations
- Schema Transformers - Transform schemas at generation
API Reference:
ModelManagerAPI - Programmatic APIModelDiffAPI - Programmatic API