Initial commit

README.md

@@ -0,0 +1,202 @@
+# Data Analysis Platform
+
+A modern full-stack data analysis platform built with Python FastAPI backend and Next.js frontend, designed for efficient data processing, visualization, and statistical analysis.
+
+## Overview
+
+This platform provides a comprehensive toolkit for data analysis workflows, combining powerful Python data science libraries with a modern, responsive web interface. It leverages Apache Arrow for high-performance data transfer and implements best practices for both backend and frontend development.
+
+### Key Features
+
+- **Backend:** FastAPI with async support, Pydantic v2 validation, and comprehensive data science stack
+- **Frontend:** Next.js 16 with TypeScript, Tailwind CSS, and Shadcn UI components
+- **Data Processing:** Pandas, Scikit-learn, and Statsmodels integration
+- **Performance:** Apache Arrow for zero-copy data transfer between services
+- **Architecture:** Feature-based frontend organization, RESTful API design
+
+## Technology Stack
+
+### Backend
+- **Python 3.12+** with FastAPI framework
+- **Pydantic v2** for schema validation
+- **Data Science:** Pandas 2.3.3+, Scikit-learn 1.8.0+, Statsmodels 0.14.6+
+- **Serialization:** Apache Arrow 22.0+ for efficient binary data transfer
+- **Package Management:** UV for fast dependency resolution
+
+### Frontend
+- **Next.js 16** (Standalone mode) with React 19
+- **TypeScript** for type safety
+- **Styling:** Tailwind CSS 4+ and Shadcn UI components
+- **Data Display:** TanStack Table, Apache Arrow 21+, Recharts
+- **State Management:** Zustand v5 for local state, TanStack Query v5 for server state
+- **Virtualization:** TanStack Virtual for handling large datasets
+
+### DevOps
+- **Docker** multi-stage builds with distroless/alpine images
+- **Docker Compose** for local development orchestration
+
+## Project Structure
+
+```
+Data_analysis/
+├── backend/                 # FastAPI backend service
+│   ├── app/                # Application modules
+│   ├── tests/              # Backend test suite
+│   ├── main.py             # Application entry point
+│   ├── pyproject.toml      # Python dependencies (UV)
+│   └── Dockerfile          # Backend container image
+│
+├── frontend/               # Next.js frontend application
+│   ├── src/
+│   │   └── features/       # Feature-based organization
+│   ├── tests/              # Frontend test suite
+│   ├── package.json        # Node.js dependencies
+│   └── Dockerfile          # Frontend container image
+│
+├── compose.yaml            # Docker Compose configuration
+├── _bmad-output/           # Planning and implementation artifacts
+└── README.md               # This file
+```
+
+## Prerequisites
+
+Before running the applications locally, ensure you have the following installed:
+
+- **Python 3.12+** - [Download Python](https://www.python.org/downloads/)
+- **Node.js 20+** - [Download Node.js](https://nodejs.org/)
+- **UV (Python package manager)** - Install via: `pip install uv`
+- **npm** (comes with Node.js)
+
+## Local Development Setup
+
+### Backend Setup
+
+1. Navigate to the backend directory:
+
+   ```bash
+   cd backend
+   ```
+
+2. Create and activate a virtual environment (optional but recommended):
+
+   ```bash
+   python3.12 -m venv .venv
+   source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+   ```
+
+3. Install dependencies using UV:
+
+   ```bash
+   uv sync
+   ```
+
+4. Start the FastAPI development server:
+
+   ```bash
+   uvicorn main:app --reload --host 0.0.0.0 --port 8000
+   ```
+
+   The backend API will be available at `http://localhost:8000`
+
+5. Access the interactive API documentation at:
+   - Swagger UI: `http://localhost:8000/docs`
+   - ReDoc: `http://localhost:8000/redoc`
+
+### Frontend Setup
+
+1. Open a new terminal and navigate to the frontend directory:
+
+   ```bash
+   cd frontend
+   ```
+
+2. Install dependencies:
+
+   ```bash
+   npm install
+   ```
+
+3. Create environment configuration file:
+
+   ```bash
+   cp .env.local.example .env.local
+   ```
+
+   Edit `.env.local` if needed to configure API endpoints or other settings.
+
+4. Start the Next.js development server:
+
+   ```bash
+   npm run dev
+   ```
+
+   The frontend application will be available at `http://localhost:3000`
+
+### Running Both Services
+
+To run both services simultaneously:
+
+1. **Terminal 1 - Backend:**
+   ```bash
+   cd backend
+   source .venv/bin/activate  # If using virtual environment
+   uvicorn main:app --reload --host 0.0.0.0 --port 8000
+   ```
+
+2. **Terminal 2 - Frontend:**
+   ```bash
+   cd frontend
+   npm run dev
+   ```
+
+## Development Workflow
+
+### Code Style and Standards
+
+- **Backend:** Follow PEP 8 guidelines with snake_case naming
+- **Frontend:** Use TypeScript strict mode, follow ESLint configuration
+- **API Convention:** Use snake_case for JSON keys to maintain consistency with Pandas DataFrames
+- **Documentation:** Include docstrings (Python) and JSDoc comments (TypeScript) for all exported functions
+
+### Testing
+
+- **Backend tests:** Run `pytest` from the `backend/` directory
+- **Frontend tests:** Run `npm test` from the `frontend/` directory (when configured)
+
+### Key Anti-Patterns to Avoid
+
+- Do NOT use standard JSON for transferring datasets larger than 5,000 rows (use Apache Arrow)
+- Do NOT use deep React Context for high-frequency state updates (use Zustand)
+- Do NOT implement opaque algorithms without logging data exclusions
+- Do NOT perform heavy blocking computations on the main FastAPI process (use background tasks)
+
+## Docker Deployment (Coming Soon)
+
+The project includes Docker configuration for containerized deployment. Instructions for running with Docker Compose will be added in the next update.
+
+## Documentation
+
+- **Project Context:** See `_bmad-output/project-context.md` for detailed implementation rules
+- **Architecture:** Technical architecture documentation is available in `_bmad-output/planning-artifacts/`
+- **API Reference:** Access interactive API documentation at `/docs` when backend is running
+
+## Contributing
+
+This project follows modern software development practices with comprehensive planning artifacts. When contributing:
+
+1. Read the project context file for implementation guidelines
+2. Follow the established code patterns and conventions
+3. Ensure all tests pass before submitting changes
+4. Update documentation as needed
+
+## License
+
+[Specify your license here]
+
+## Support
+
+For questions or issues related to this project, please refer to the project documentation or contact the development team.
+
+---
+
+**Last Updated:** 2026-01-11