Cleanifix

A CLI tool that automatically cleans your data files through natural language commands. Like having a data analyst in your terminal.

🚀 Quick Start bash# Install npm install -g cleanifix

Basic usage

cleanifix @sales.csv "remove duplicates" cleanifix @users.csv "fill missing emails with 'unknown@example.com'" cleanifix @data.json "standardize all dates to ISO format"

Interactive mode

cleanifix interactive @messy_data.csv 🎯 Features Core Capabilities (MVP)

Missing Value Detection & Handling - Find and fix missing data automatically Data Standardization - Normalize dates, phone numbers, addresses, and more Deduplication - Remove duplicate rows with smart matching

Natural Language Interface bash# Just describe what you want cleanifix @customers.csv "find missing phone numbers and fill with 'N/A'" cleanifix @inventory.csv "standardize product names to title case" cleanifix @transactions.csv "remove duplicate entries keeping the most recent" Smart Suggestions bash$ cleanifix @data.csv "analyze"

📊 Data Quality Report: ✗ 156 missing values in 'email' column ✗ 89 inconsistent date formats ✗ 34 potential duplicates

Suggested fixes:

1. Fill missing emails with domain-based patterns
2. Standardize dates to YYYY-MM-DD
3. Remove exact duplicates keeping first occurrence

Apply all fixes? [Y/n] 📦 Installation Prerequisites

Node.js 18+ Python 3.8+ 4GB RAM recommended for large files

Install from npm bashnpm install -g cleanifix Install from source bashgit clone https://github.com/rickyjs1955/cleanifix.git cd cleanifix ./scripts/setup-dev.sh 🛠️ Usage Examples Basic Cleaning bash# Find issues cleanifix @data.csv "show me data quality issues"

Fix missing values

cleanifix @sales.csv "fill missing prices with median"

Standardize formats

cleanifix @contacts.csv "standardize all phone numbers to international format"

Remove duplicates

cleanifix @emails.csv "remove duplicate emails keeping the latest entry" Batch Processing bash# Create a config file cat > cleaning-rules.yaml << EOF rules:

• type: missing_values columns: [price, quantity] strategy: median
• type: standardize column: phone format: E164
• type: deduplicate keys: [email] keep: last EOF

Run batch cleaning

cleanifix batch @data.csv --rules cleaning-rules.yaml Interactive Mode bashcleanifix interactive @messy_data.csv

🧹 Cleanifix Interactive Mode

analyze my data fill missing ages with average by city
standardize all names to proper case save as cleaned_data.csv exit 🏗️ Architecture Cleanifix uses a hybrid architecture:

CLI Interface (Node.js) - Fast, responsive user interaction Processing Engine (Python) - Powerful data manipulation with pandas Communication - JSON-based message passing between components

🤝 Contributing We welcome contributions! See CONTRIBUTING.md for guidelines. Development Setup bash# Clone the repo git clone https://github.com/rickyjs1955/cleanifix.git cd cleanifix

Setup development environment

./scripts/setup-dev.sh

Run tests

npm test # CLI tests python -m pytest # Engine tests

Run in development mode

npm run dev 📋 Roadmap Phase 1 (Current) - MVP

Basic CLI interface Missing value handling Simple standardization Exact deduplication CSV support JSON support

Phase 2 - Enhanced Rules

Fuzzy deduplication Custom regex patterns Outlier detection Data type inference Excel support

Phase 3 - ML Integration

Smart imputation Anomaly detection Pattern learning Confidence scoring Auto-cleaning mode

📄 License MIT License - see LICENSE file for details 🙏 Acknowledgments Built with:

Commander.js - CLI framework Pandas - Data manipulation Chalk - Terminal styling

💬 Support

Documentation: docs.cleanifix.dev Issues: GitHub Issues Discussions: GitHub Discussions

Made with ❤️ by data people, for data people