MERN Web Scrapper

A web application that scrapes data from websites and displays it in a user-friendly interface. The project is built using the MERN stack (MongoDB, Express.js, React.js, Node.js).

Table of Contents

• Features
• Technologies Used
• Setup Instructions
• Usage
• Future Enhancements
• Contributing

Features

• Data Scraping: Scrapes data from specified URLs.
• Responsive UI: Displays data in a clean, professional UI.
• Error Handling: Handles failed requests and displays user-friendly error messages.

Future Updates

• Extract more information from various pages of the University.
• Improve data fetching time.
• Impove UI.

Technologies Used

• Frontend: React.js, Tailwind CSS
• Backend: Node.js, Express.js
• Database: MongoDB
• Web Scraping: Cheerio.js or Puppeteer

Demo ScreenShots

This content Gets dynamically update everytime.

Setup Instructions

1. Clone the repository:

   git clone https://github.com/THEYASHGAUR/MERN-Web-Scrapper.git

2. Navigate to the project directory:

   cd MERN-Web-Scrapper

3. Install dependencies:

   npm install

4. Set up environment variables for MongoDB connection.
5. Start the development server:

   npm run dev

Usage

1. Enter the URL of the website you want to scrape.
2. Click on the "Scrape" button.
3. View the scraped data in the interface.

Future Enhancements

• Authentication: Add user authentication for personalized data storage.
• Advanced Scraping: Implement advanced scraping techniques to handle dynamic websites.
• Pagination: Add pagination for displaying large datasets.
• Export Data: Enable exporting of scraped data in CSV or JSON format.
• Multi-language Support: Add support for multiple languages.

Contributing

1. Fork the repository.
2. Create a new branch (git checkout -b feature-branch).
3. Commit your changes (git commit -m 'Add new feature').
4. Push to the branch (git push origin feature-branch).
5. Create a pull request.