Prueba de concepto de un sistema automatizado de recolección y análisis de datos de compra y venta de viviendas en Amazon web Service.
Visualitza/Obre
Estadístiques de LA Referencia / Recolecta
Inclou dades d'ús des de 2022
Cita com:
hdl:2117/374756
Tipus de documentTreball Final de Grau
Data2022-09-05
Condicions d'accésAccés obert
Llevat que s'hi indiqui el contrari, els
continguts d'aquesta obra estan subjectes a la llicència de Creative Commons
:
Reconeixement-NoComercial-SenseObraDerivada 3.0 Espanya
Abstract
This work has been done because of my interest in some techniques used to collect information from web pages. These techniques have a name and it is called web scraping. Web scraping is used to crawl information on web pages. This information can then be used to draw conclusions from the data that otherwise could not be drawn. For example, web scraping is used to track down car parts that are no longer for sale on used car parts portals. As these parts are in short supply, they are quickly sold. With web scraping it is possible to receive a notification when one of these parts is offered for sale on one of the portals. The work done is a proof of concept of web scraping with the Scrapy framework. Scrapy is a framework written in Python used to create bots that collect information from websites. During the work a bot is made with Scrapy. Before the project I didn't know that the traffic of bots on the internet was so high, about 66%. It is for this reason that many websites do not allow these bots to browse their pages by filtering their traffic. The project explains what modifications have been made to the bot so that it can bypass these website defenses. This project could be used to help websites improve their defenses against traffic from bots that only want to collect information from their pages. The project has also deployed the application on the web service provider amazon web services. Amazon web service is one of the largest web service providers in the world. There are many job offers that require knowledge of this provider and it seemed interesting to me to know how applications could be deployed on it.
TitulacióGRAU EN ENGINYERIA DE SISTEMES DE TELECOMUNICACIÓ (Pla 2009)
Fitxers | Descripció | Mida | Format | Visualitza |
---|---|---|---|---|
memoria.pdf | 956,8Kb | Visualitza/Obre |