Sources for Data-Mining: Global Data

These are large, open data repositories hosting academic, governmental or even personal datasets. Each repository may host up to thousands of datasets, but users are advised to pay attention to the credibility of individual dataset providers.

Google Dataset Search

https://datasetsearch.research.google.com/    FREE 

figshare

https://figshare.com/browse/    FREE 

Mendeley Data

https://data.mendeley.com/research-data/    FREE 

World Bank Open Data

https://data.worldbank.org/    FREE 

Collected and managed by World Bank, data sets on World Bank Open Data are very diverse with an intended focus on people's well-being. Data sets can be pulled out by country, topic or indicator, with various download options.

• Agriculture & rural development, Climate change...
• External debt, Infrastructure, Social development...
• Gender issues, Health, Poverty...

UNdata

http://data.un.org    FREE 

Managed by the United Nations, UNdata serves as a single point to retrieve selected data collected by UN's numerous departments. After clicking "Datamarts," you can retrieve data sets by topic or departmental source.

• Crime, Education, Refugees...
• Energy, Environment, Food and Agriculture...
• Health, HIV/AIDS, Tourism...

Nevertheless, individual UN departmental websites oftentimes provide much more data on relevant topics. For example:

   ESCAP Online Statistical Database  (http://data.unescap.org/escap_stat/)    FREE 

• Managed by Economic and Social Commission for Asia and the Pacific of the United Nations, ESCAP provides a regional perspective on development issues in Asia and the Pacific. Data comparison between countries is available. Hong Kong and Macau data are also included.

   FAOSTAT  (http://www.fao.org/faostat/en/)    FREE 

• Managed by Food and Agriculture Organization of the United Nations, FAOSTAT provides extensive food and agriculture data for over 245 countries and territories from 1961. Data comparison between countries are also available.

   UN Comtrade Database  (https://comtrade.un.org/)    FREE 

• Managed by Comtrade Labs of the United Nations, UN Comtrade provides free access to extensive official international trade statistics and relevant analytical tables.

Being a digital project managed by the University of Oxford, Our World in Data takes a scholarly approach to data collection, sharing and discussion. Through exploring various topics, the website aims to give a global overview of changes in living conditions over a very long run.

• Population, Health, Food...
• Work & life, War & peace, Violence & rights...
• Technology, Media, Culture...

ICPSR: Find & Analyze Data

https://www.icpsr.umich.edu/icpsrweb/ICPSR/    FREE 

Different from ordinary public data archives, this data archive managed by Inter-university Consortium for Political and Social Research reviews and enhances deposited data sets before making them publicly available. This archive contains more than 250,000 files of research in the social and behavioral sciences.

Statista

https://www-statista-com.lib-ezproxy.hkbu.edu.hk/    SUBSCRIBED BY LIBRARY  

Statista provides access to statistics gathered by market researchers, trade organizations, scientific publications, and governments on over 600 industries. It is a comprehensive portal for business and industry statistical data.

Wharton Research Data Services

https://wrds-web.wharton.upenn.edu/wrds/    FREE 

Managed by the University of Pennsylvania, WRDS provides extensive data on accounting, banking, economics, and finance. Although data is freely available to the public, users have to register for an account in order to get access to the data.

ILOSTAT

https://www.ilo.org/ilostat    FREE 

Managed by the International Labour Organization, ILOSTAT provides global labor data including labor force, unemployment, working time, safety and health at work.

IMF Data

https://www.imf.org/en/data    PARTIALLY FREE 

Managed by International Monetary Fund, IMF Data provides various types of global financial data or statistics. However, not all data sets are freely available online. Highlighted free data sets include International Financial Statistics, Government Finance Statistics, and Direction of Trade Statistics. 

World Press Trends Database

http://www.wptdatabase.org.lib-ezproxy.hkbu.edu.hk/    SUBSCRIBED BY LIBRARY  

WPTD provides a range of data on the newspaper industry worldwide, including circulation, readership, revenue, advertising expenditure, etc. in each country. Go to "Custom Reports" for data that goes back to 2000.

WARC: Data

https://www-warc-com.lib-ezproxy.hkbu.edu.hk/data     SUBSCRIBED BY LIBRARY  

WARC gives data on advertising expenditure, media consumption, and media costs by channel and country. Only statistical reports and charts are provided without raw data.

RCSB Protein Data Bank

https://www.rcsb.org/    FREE 

Co-managed by Rutgers University and University of California, San Diego, RCSB PDB is an open access digital data resource in biology and medicine, providing access to over 147,000 sets of 3D structure data for large biological molecules.

Look Out for Semester-Based
Data Software Training!

Other Relevant Libgudes:

Feel free to contact me if you have questions about
Research Data Services.

Rebekah Wong
Head, Digital & Multimedia Services
rebekahw@hkbu.edu.hk