Scribd Logo
Upload

Welcome to Scribd!

  • Submitted by Keerthana Subramani ASU ID: 1203845157

    Uploaded by

    Keerthana Subramanian
    94 views4 pages
    The document describes how to crawl the Huffington Post news website for articles and user comments. It involves specifying a seed URL and search query, then crawling URLs using depth-first search and storing data in a database. The crawled pages are analyzed to extract the article title, timestamp, tags and text to store in a database, along with any user comments. The stored data can then be queried by timestamp to create a timeline visualizing the dates and titles of events over time.

    Original Description:

    arab spring data collection

    Original Title

    arab spring

    Copyright

    © Attribution Non-Commercial (BY-NC)

    Available Formats

    PPTX, PDF, TXT or read online from Scribd

    Did you find this document useful?

    Is this content inappropriate?

    Report this Document
    The document describes how to crawl the Huffington Post news website for articles and user comments. It involves specifying a seed URL and search query, then crawling URLs using depth-first search and storing data in a database. The crawled pages are analyzed to extract the article title, timestamp, tags and text to store in a database, along with any user comments. The stored data can then be queried by timestamp to create a timeline visualizing the dates and titles of events over time.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPTX, PDF, TXT or read online from Scribd
    Flag for inappropriate content
    94 views4 pages

    Submitted by Keerthana Subramani ASU ID: 1203845157

    Uploaded by

    Keerthana Subramanian
    The document describes how to crawl the Huffington Post news website for articles and user comments. It involves specifying a seed URL and search query, then crawling URLs using depth-first search and storing data in a database. The crawled pages are analyzed to extract the article title, timestamp, tags and text to store in a database, along with any user comments. The stored data can then be queried by timestamp to create a timeline visualizing the dates and titles of events over time.

    Copyright:

    Attribution Non-Commercial (BY-NC)
    Download as PPTX, PDF, TXT or read online from Scribd
    Flag for inappropriate content
    of 4

    Submitted by Keerthana Subramani ASU ID: 1203845157

    Source: Huffington Post News website that has news articles and user comments. Data type: Text data Structure of pages: HTML Seed URL and search query is specified Crawl URLs using depth first approach and store in database. k, k+1th , k+2 and so on..

    Eyeballing Identify anchors Access source using the URL Run RE over HTML data Get title, timestamp, tags and article data and store to database. User comments are also extracted. Identify by string matching

    Query dataset using timestamp. Creating a timeline of events from the timestamp of each data Software used: Timeline creator Shows the date and title, occurrence of events over time

    You might also like