Introduction (Fall 2018)
3 minute read
Introduction to Big Data Applications
This is an overview course of Big Data Applications covering a broad range of problems and solutions. It covers cloud computing technologies and includes a project. Also, algorithms are introduced and illustrated.
General Remarks Including Hype cycles
This is Part 1 of the introduction. We start with some general remarks and take a closer look at the emerging technology hype cycles.
1.a Gartner’s Hypecycles and especially those for emerging technologies between 2016 and 2018
.
1.b Gartner’s Hypecycles with Emerging technologies hypecycles and the priority matrix at selected times 2008-2015
- .
1.a + 1.b:
- Technology trends
- Industry reports
Data Deluge
This is Part 2 of the introduction.
2.a Business usage patterns from NIST
- .
2.b Cyberinfrastructure and AI
- .
2.a + 2.b
- Several examples of rapid data and information growth in different areas
- Value of data and analytics
Jobs
This is Part 3 of the introduction.
- .
.
- Jobs opportunities in the areas: data science, clouds and computer science and computer engineering
- Jobs demands in different countries and companies.
- Trends and forecast of jobs demands in the future.
Industry Trends
This is Part 4 of the introduction.
4a. Industry Trends: Technology Trends by 2014
4b. Industry Trends: 2015 onwards
An older set of trend slides is available from:
4a. Industry Trends: Technology Trends by 2014
A current set is available at:
4b. Industry Trends: 2015 onwards
- .
4c. Industry Trends: Voice and HCI, cars,Deep learning
- Many technology trends through end of 2014 and 2015 onwards, examples in different fields
- Voice and HCI, Cars Evolving and Deep learning
Digital Disruption and Transformation
This is Part 5 of the introduction.
- Digital Disruption and Transformation
- .
.
- The past displaced by digital disruption
Computing Model
This is Part 6 of the introduction.
6a. Computing Model: earlier discussion by 2014:
- .
.
6b. Computing Model: developments after 2014 including Blockchain:
- .
.
- Industry adopted clouds which are attractive for data analytics, including big companies, examples are Google, Amazon, Microsoft and so on.
- Some examples of development: AWS quarterly revenue, critical capabilities public cloud infrastructure as a service.
- Blockchain: ledgers redone, blockchain consortia.
Research Model
This is Part 7 of the introduction.
Research Model: 4th Paradigm; From Theory to Data driven science?
- .
.
- The 4 paradigm of scientific research: Theory,Experiment and observation,Simulation of theory or model,Data-driven.
Data Science Pipeline
This is Part 8 of the introduction. 8. Data Science Pipeline
- .
- DIKW process:Data, Information, Knowledge, Wisdom and Decision.
- Example of Google Maps/navigation.
- Criteria for Data Science platform.
Physics as an Application Example
This is Part 9 of the introduction.
- Physics as an application example.
Technology Example
This is Part 10 of the introduction.
- Overview of many informatics areas, recommender systems in detail.
- NETFLIX on personalization, recommendation, datascience.
Exploring Data Bags and Spaces
This is Part 11 of the introduction.
- Exploring data bags and spaces: Recommender Systems II
- .
- Distances in funny spaces, about “real” spaces and how to use distances.
Another Example: Web Search Information Retrieval
This is Part 12 of the introduction. 12. Another Example: Web Search Information Retrieval
- .
Cloud Application in Research
This is Part 13 of the introduction discussing cloud applications in research.
- Cloud Applications in Research: Science Clouds and Internet of Things
Software Ecosystems: Parallel Computing and MapReduce
This is Part 14 of the introduction discussing the software ecosystem
- Software Ecosystems: Parallel Computing and MapReduce
Conclusions
This is Part 15 of the introduction with some concluding remarks. 15. Conclusions
- .
.