Tools to support TDM

Using APIs to text and data mine

An Application Programming Interface (API) is a digital intermediary that allows different software applications to communicate and share data with one another.

When used for text and data mining, an API allows researchers to fetch larger volumes of data than could be achieved via a standard web browser. An API often provides machine readable data for computational analysis in the form of texts, metadata or images. Some content providers offer a free API as part of the Library’s subscription, but most require researchers to complete a separate application process to gain access to the API. They may ask for a supporting letter from the University Library to verify that we have an active subscription to the resource. You can contact the Library to discuss this further. 

Whilst many content suppliers offer a basic free TDM API as part of our institutional licence, they may apply limits. For example, limits on download or throttle rates, limits on the number of items that can be downloaded at any one time, and limits on intended use. They may provide less restrictive options at an additional cost which cannot usually be funded by the Library.  

University of Sheffield training and support

Services and training we offer to our researchers and students include:

• Data analysis service (login required): A Research IT service. They offer one-to-one consultations to advise on data analysis projects and can also collaborate on projects if included in the research funding application. 
• Research Computing Training : Research IT offer a range of training options including Python, Matlab, R and Machine Learning.
• R workshops (login required): Beginners programming with R with the MASH team.

TDM tools and software

Some no-code text and data mining tools include:

• KNIME: This can also be downloaded from the University of Sheffield software center (on the Start menu) for managed PCs.  Open source data analysis tool with application to text and data mining techniques.
• SPSS Modeler Premium (login required - limited to University of Sheffield licence):  builds predictive models and conducts other analytical tasks. Contains text analytics functionality.  
• Orange:  Open source machine learning and data visualisation with data analysis workflows.  
• Altair Rapid Miner: Text and data mining tool. Free education licence for a year and unlimited free licence for a limited amount of data (10,000 lines).
• Weka: A collection of visualisation tools and algorithms for data analysis and predictive modelling created by the University of Waikato, New Zealand, available under a free software licence. Companion software for the text: Data mining: practical machine learning tools and techniques, by Ian H. Witte (find this text at Sheffield - login required).