CIS 606 PU Computer Science API Analysis and Data Extraction Essay

Introduction

Data that you may want to use in a data mining project may come from different sources, it may also be in different formats. Those sources may be static (data is not changing) or dynamic when the time of data acquisition is important as data in the same source may change. A common example of dynamic sources is API. Besides various ways to use them for systems interfaces, APIs can be seen as sources of data when a certain type of request (e.g. GET type in RESTfull APIs) is executed. Note that many APIs provide data in JSON format.

Directions

Pick any API from the list:

https://github.com/public-apis/public-apisLinks to an external site.

which may potentially be meaningful for your field of study (e.g. business). Note, that some APIs require a complex authentication while others don’t (Auth: NO). Also, select an API that would represent data that changes at least daily by its nature (e.g. exchange rates may change daily, while Uber rides data change very frequently).

Load sample data from the selected API at least two times (chose an interval between those requests based on the pace of the data change). Create a single .CSV file that would include all the data provided by the API in two or more requests. Make sure to label each record provided by an API with the time of the request. Also, include column(s) that will show the difference of at least one value that occurred between the requests.

For example, if a Weather API returns a JSON file with the current temperature in 10 cities, convert data to .CSV, which when combined would have at least  20 records (2 for each city based on two requests), add an additional column with the time of the request, and a column showing the difference between the temperature during the second and the first request.