Welcome to DATAMIMIC, the Model-Driven and AI-powered platform that revolutionizes test data generation! By leveraging advanced AI and model-driven technologies, DATAMIMIC enables developers and testers to create realistic, scalable, and privacy-compliant test data with ease.
Available in Both Community and Enterprise Editions
- Create sophisticated data models for consistent test data generation
- Define complex relationships between entities
- Support for nested and hierarchical data structures
- Basic Data Types Support
- All standard primitive types
- Complex data structures
- Custom data type definitions
- Core Database Integration
- RDBMS support (PostgreSQL, MySQL, Oracle)
- MongoDB integration
- Basic import/export functionality
- GDPR-compliant data anonymization
- Basic pseudonymization capabilities
- Data masking for sensitive information
- Configurable privacy rules
- High Performance Engine
- Optimized for large datasets
- Parallel processing support
- Memory-efficient operations
- Python Integration
- Native Python API
- Seamless dependency management
- Python script extensions
- Basic Extensibility
- Custom generator support
- Plugin architecture
- Basic scripting capabilities
Exclusive to Enterprise Edition
Advanced AI capabilities for realistic data creation
- GAN-based Synthesis
- Realistic data patterns
- Learning from existing datasets
- Pattern replication
- LLM Integration
- Natural language content
- Context-aware generation
- Semantic consistency
- Automatic Modeling
- Schema inference
- Pattern detection
- Model optimization
Enterprise-grade system integration capabilities
- Streaming Support
- Kafka integration
- Real-time data generation
- Stream processing
- Enterprise Formats
- EDI processing
- Advanced XSD handling
- Custom format support
- Advanced Connectors
- Enterprise systems
- Cloud platforms
- Legacy systems
Advanced data protection and compliance
- Advanced Anonymization
- Context-aware masking
- Reversible anonymization
- Custom privacy rules
- Compliance Tools
- Audit logging
- Compliance reporting
- Policy enforcement
- Enterprise Security
- Role-based access
- Encryption support
- Security audit trails
Comprehensive data quality assurance
- Validation Framework
- Custom rule engines
- Complex validation logic
- Cross-field validation
Traditional test data generation can be time-consuming and may compromise data privacy. DATAMIMIC addresses these challenges by:
- Reducing Time-to-Market: Quickly generate test data without manual intervention.
- Enhancing Test Coverage: Simulate diverse data scenarios for comprehensive testing.
- Ensuring Compliance: Maintain data privacy and comply with legal regulations.
- Improving Data Quality: Generate realistic data that mirrors production environments.
- Operating System: Windows, macOS, or Linux
- Python: Version 3.10 or higher
- Optional: uv Package Manager for development setup GitHub
The simplest way to get started with DATAMIMIC is through pip:
pip install datamimic-ceVerify the installation:
datamimic --versionFor contributors and developers who want to work with the source code:
-
Install uv Package Manager:
pip install uv
-
Clone and set up the repository:
git clone https://github.com/rapiddweller/datamimic.git cd datamimic uv sync
-
Create a new data generation project:
datamimic init my-project cd my-project -
Configure your data model in
datamimic.xml:<setup> <generate name="datamimic_user_list" count="100" target="CSV,JSON"> <variable name="person" entity="Person(min_age=18, max_age=90, female_quota=0.5)"/> <key name="id" generator="IncrementGenerator"/> <key name="first_name" script="person.given_name"/> <key name="last_name" script="person.family_name"/> <key name="gender" script="person.gender"/> <key name="birth_date" script="person.birthdate" converter="DateFormat('%d.%m.%Y')"/> <key name="email" script="person.family_name + '@' + person.given_name + '.de'"/> <key name="ce_user" values="True, False"/> <key name="ee_user" values="True, False"/> <key name="datamimic_lover" constant="DEFINITELY"/> </generate> </setup>
-
Generate data:
datamimic run datamimic.xml
-
Access the generated data in the
outputdirectory.json export:
[ {"id": 1, "first_name": "Mary", "last_name": "Mcgowan", "gender": "female", "birth_date": "1946-05-15T00:00:00", "email": "[email protected]", "ce_user": false, "ee_user": true, "datamimic_lover": "DEFINITELY"}, {"id": 2, "first_name": "Gabrielle", "last_name": "Malone", "gender": "female", "birth_date": "1989-11-27T00:00:00", "email": "[email protected]", "ce_user": false, "ee_user": true, "datamimic_lover": "DEFINITELY"}, {"id": 4, "first_name": "Margaret", "last_name": "Torres", "gender": "female", "birth_date": "2006-07-13T00:00:00", "email": "[email protected]", "ce_user": false, "ee_user": false, "datamimic_lover": "DEFINITELY"}, {"id": 5, "first_name": "Monica", "last_name": "Meyers", "gender": "female", "birth_date": "1983-07-22T00:00:00", "email": "[email protected]", "ce_user": true, "ee_user": false, "datamimic_lover": "DEFINITELY"}, {"id": 6, "first_name": "Jason", "last_name": "Davis", "gender": "male", "birth_date": "1941-07-05T00:00:00", "email": "[email protected]", "ce_user": true, "ee_user": false, "datamimic_lover": "DEFINITELY"}, {"...": "..."}, {"id": 100, "first_name": "Jared", "last_name": "Rivas", "gender": "male", "birth_date": "1975-03-16T00:00:00", "email": "[email protected]", "ce_user": true, "ee_user": true, "datamimic_lover": "DEFINITELY"} ]csv export:
id|first_name|last_name|gender|birth_date|email|ce_user|ee_user|datamimic_lover 1|Mary|Mcgowan|female|1946-05-15 00:00:00|[email protected]|False|True|DEFINITELY 2|Gabrielle|Malone|female|1989-11-27 00:00:00|[email protected]|False|True|DEFINITELY 3|Antonio|Davis|male|2005-05-12 00:00:00|[email protected]|False|True|DEFINITELY 4|Margaret|Torres|female|2006-07-13 00:00:00|[email protected]|False|False|DEFINITELY 5|Monica|Meyers|female|1983-07-22 00:00:00|[email protected]|True|False|DEFINITELY ... 100|Jason|Davis|male|1941-07-05 00:00:00|[email protected]|True|False|DEFINITELY
DATAMIMIC supports various advanced features:
- Custom Generators: Create your own data generators
- Data Relationships: Define complex relationships between entities
- Import/Export Formats: Support for JSON, XML, CSV, RDBMS and MongoDB
- Import/Export Formats ( only EE ): Kafka, EDI, XSD and more
- Data Anonymization: Anonymize data to comply with privacy regulations
- Data Validation: Define and enforce data validation rules
- Scripting: Extend functionality using Python scripts
- Database Integration: Connect to databases for seamless data generation
- Model-Driven Generation: Utilize models to generate realistic data
- Validation Rules: Define and enforce data validation rules
- Scripting: Extend functionality using Python scripts
Explore our demo collection:
# List available demos
datamimic demo list
# Run a specific demo
datamimic demo create demo-model
datamimic run ./demo-model/datamimic.xmlKey demos include:
- Basic entity generation
- Complex relationships
- Database integration
- Custom generator creation
- Privacy compliance examples
Find more examples in the datamimic_ce/demos directory.
We ❤️ contributions! Here's how you can help:
- Code Contributions: Submit pull requests for new features or bug fixes.
- Documentation: Improve existing docs or help with translations.
- Community Engagement: Join discussions and support other users.
Open Source Freedom for Everyone
- 🔓 MIT License: Maximum freedom for innovation
- 💼 Commercial Ready: Use freely in commercial projects
- 🔄 Modification Rights: Full source code access and modification rights
- 🌍 No Restrictions: Deploy anywhere, anytime
-
📦 Core Features
- Model-driven data generation
- Basic data types & integrations
- GDPR compliance tools
-
👥 Community Support
- Active GitHub community
- Public issue tracking
- Community discussions
- Regular updates
- Individual developers
- Startups & small teams
- Open source projects
- Learning & evaluation
- POC development
Professional Power for Business Success
- 📋 Commercial License: Enterprise-grade flexibility
- 🔐 Advanced Features: Full suite of professional tools
- 🎯 Priority Support: Direct access to expert team
- 🛠️ Custom Solutions: Tailored to your needs
-
🤖 AI Capabilities
- GAN-based synthesis
- LLM integration
- Automated modeling
-
🔗 Enterprise Integration
- Advanced connectors
- Kafka streaming
- EDI support
-
🛡️ Enhanced Security
- Advanced privacy features
- Compliance reporting
- Audit trails
- Large enterprises
- Financial institutions
- Healthcare organizations
- Government agencies
- High-compliance industries
Ready to unlock the full potential of DATAMIMIC?
Contact Our Team:
- 📧 Email: [email protected]
- 🌐 Visit: datamimic.io/enterprise
| Feature | Community | Enterprise |
|---|---|---|
| Base Features | ✅ | ✅ |
| Source Code Access | ✅ | ✅ |
| Commercial Use | ✅ | ✅ |
| AI Features | ❌ | ✅ |
| Priority Support | ❌ | ✅ |
| Enterprise Integrations | ❌ | ✅ |
| SLA Support | ❌ | ✅ |
| Custom Development | ❌ | ✅ |
"Empower your data generation journey with the right DATAMIMIC edition for your needs"
Need help or have questions? We're here for you!
Stay updated and connect with our community!
- 🌐 Website: www.datamimic.io
- 🏢 Rapiddweller: www.rapiddweller.com
- 💼 LinkedIn: rapiddweller
A big thank you to all our contributors! Your efforts make DATAMIMIC possible.
Don't forget to ⭐ star and 👀 watch this repository to stay updated!
Legal Notices
For detailed licensing information, please see the LICENSE file.