[K-MOOC] Instruction to Deep Learning: 1-2. Methodology of Machine Learning

Machine Learning is so Extensive and Sophisticated

Machine Learning Tasks

1. Classification
1. to classify the data into specific category
2. categories are pre-assigned
   
2. Regression
1. Linear regression
1. to find linear function that explain independent variable x and dependent variable y in given data set, {(x, y)}
2. simple linear regression:
   y-hat = f(x) = 𝜷₀ + 𝜷₁𝑿𝑖
2. Logistic regression
1. is different to linear regression in that dependent variable(y) is nominal type
2. is a kind of stochastic elements
   
3. Clustreing
1. is almost the same to the classification, but the only one difference is that is has no certain categorues
2. to make cluster by learning characters of independent data (not need training)
3. application cases
1. to classify document: by frequency of words
2. to classify satellite pictures: by color tone

Machine Learning Modes

1. Supervised learning
1. teacher + students
2. classification, regression
2. Unsupervised learning
1. is trained with unlabeled data (without answer)
2. clustering
3. Reinforcement learning
1. was invented in 1990s, and is spotlighted in these days
2. is learning method that makes computer plays better than human being
3. to mapping the state to an specific action which brings the best reward

Machine Learning Technique

[K-MOOC] Instruction to Deep Learning: Syllabus

• Course: Instruction to Deep Learning
• Professor: Hee-chul Kim / Daegu University
• Goals
• Understanding principle of deep learning and algorithm
• Prerequisite
• Mathematics(matrix, vector, differential), Computer
• Schedule
• 1 week: Understanding of machine learning
• 1-1. Outline of Machine Learning
• 1-2. Methodology of Machine Learning

[K-MOOC] Instruction to Deep Learning: 1-1. Outline of machine learning

Definition of Artificial Intelligence

1. the state that machine has intelligence (Nils Nilsson, 2010, 'The quest of AL')
2. but the problem is that the intelligence is ambiguous
   

Practical Definition of AI

1. the technology that machine could carry out a process in smart way
2. whole range of research of AI scientist (Stanford AI 100 years Report)
3. the complex of all elements of recognition process in human being
   

History of AI

1. 1956s: 
1. a first use of the term, 'Artificial Intelligence', 
2. meaning of 'to proceed on the basis of the conjecture that every of learning or any other feature of intelligence can in principle be so precisely described that a machine can be made to simulate it' (on Dartmouth College Workshop)
   
   
   
2. 1970s, 1990s: First AI winter, even though huge amount of investments and financial supports is gathered, the outcome was none and all those supports were cut off
   
3. 2010s: the interest and expectation to AI is growing
   
   

Relationship of AI, ML, DL

Definition of Machine Learning

1. the completely different method from the conventional programming method
1. programming method: data → program → output
2. Machine learning: data & output → algorithm → program
2. a detailed field of AI that functions intellectually after a computer is learning from its experience
3. in mathematical meaning,
1. y = h(x)
   y: output
   h: function
   x: feature
2. is finding the function h, h( ), which is the closest to target function by using a set of sample, S={(x, y)}
   

What is the Deep Learning?

1. a multilayered structure of Neural Networks (large scale)
2. is being trained by hierarchical abstract learning

Benefits of DL

1. End-to-End learning: just give input data and get output

Phi X 174

What is it?

1. a single-stranded DNA(ssDNA) virus that infects Escherichia coli
2. the first DNA-based genome to be sequenced in 1977
3. Well-defined, small(5,396bp), and diverse(45% GC, 55% AT) genome
4. fasta file download link:
1. PhiX_from_Illumina
2. PhiX_from_NCBI
5. Using it as a positive control in Illumina NGS

What are benefits of using PhiX control?

1. Calibration Control: can be run alone and serves as a calibration control for;
1. Cluster generation: can be used as a positive control in the clustering process
   
   

   Platform	Mode/Reagents	Optimal Raw Cluster Density
   HiSeq	High Output, TruSeq v3	750-850 K/mm²
   	High Output, HiSeq v4 (required upgrade)	950-1050 K/mm²
   	Rapid v2	850-1,000 K/mm²
   MiSeq	v2	1,000-1,200 K/mm²
   	v3	1,200-1,400 K/mm²
   MiniSeq	Mid and High Output	170-220 K/mm²
   NextSeq	Mid and High Output, v2	170-220 K/mm²

   [table 1] Cluster density guidelines for Illumina sequencing platforms
   
   
2. Cross talk matrix generation
1. During an illumina sequencing run, the cross-talk due to spectral overlap between the 4 fluorescently labeled nucleotides is calculated during template generation in cycle 1-5
2. https://www.slideshare.net/idtdna/unique-dualmatched-adapters-mitigate-index-hopping-between-ngs-samples
3. Phasing and Prephasing
1. During sequencing by synthesis, each DNA strand in a cluster extends by 1 base per cycle
2. A small proportion of strands may become out of phase with the current cycle, either falling a base behind(phasing) or jumping a base ahead(prephasing)
3. For best results, use a PhiX spike-in as a control with any library that does not comprise a balanced base composition
4. High GC samples(≧ 60%) typically show higher phasing rates, and in this case a PhiX control is required


2. Run quality monitor: due to its small size and balanced nucleotide composition, it's an ideal in-run control (typically with >= 1% spike-in) for run quality monitoring
   
   

   Platform	PhiX Aligned(%)
   iSeq 100	minimum 5%
   MiniSeq	10~50%
   MiSeq (MCS 2.2 or higher)	minimum 5%
   NextSeq	10~50%
   HiSeq 2500 (HCS 2.2.38 or higher)	minimum 10%
   HiSeq 3000/4000 (HCS 3.3.76 or lower)	10~50%
   HiSeq 3000/4000 (HCS 3.4.0 or higher)	5~20%
   NovaSeq	minimum 10%

   [table 2] PhiX Control v3 library Illumina recommends spiking in when running low diversity libraries
   
   
3. Color balancing
1. For low diversity libraries, the PhiX Control v3 library provides balanced fluorescent signals at each cycle to improve the overall run quality
2. You can find why the nucleotide diversity is important in here

How to remove PhiX reads from the fastq

Nucleotide Diversity

Nucleotide diversity는 무엇인가요?

1. High nucleotide diversity: Library가 sequencing의 모든 cycle에서 4개의 nucleotides를 거의 동등한 비율로 골고루 가지고 있는 상태를 의미합니다.
2. 아래 그림에서 well-balanced 상태와 unbalanced 상태의 diversity와 base-balance를 나타내고 있습니다. 그리고 이런 상태가 Sequencing Analysis Viewer(SAV)의 % base plot에서 어떻게 반영되어 나타나는지 보여주고 있습니다.

[fig 1] Illustrates of the diversity and base-balance

Nucleotide diversity는 왜 중요한가요?

1. Nucleotide diversity는 효과적인 template 생성에 필요하며 high-quality data 생산에 중요합니다.
2. Diversity는 MiniSeq, MiSeq, NextSeq, 그리고 HiSeq 1000-2500 system에서 첫 번째 sequencing read의 4~7 cycles 동안 중요합니다. Sequencing software는 template generation이라 불리는 과정에서 이런 앞 부분 cycles의 image들을 사용하여 각 cluster의 위치를 확인합니다.
3. Diversity는 또한 첫 25 cycle에서도 중요한데, phasing/pre-phasing, color matrix corrections, 그리고 pass filter calculations가 일어나는 과정이기 때문입니다.
4. Real-Time Analysis(RTA) software는 적절한 양의 PhiX 투입이 필요합니다. 이와 관련된 내용은 here에서 찾아볼 수 있습니다.

ref)

https://support.illumina.com/bulletins/2016/07/what-is-nucleotide-diversity-and-why-is-it-important.html

[K-MOOC] Data Analytics for Forecasting and Classification: Syllabus

• Course: Data Analytics for Forecasting and Classification
• Professor: Chi-hyuk Jeon / POSTECH
• Goals
• Understanding data analysis methods for forecasting and classification based on statistics
• Cultivating data analysis skill and application ability by using data analytics methods
• Prerequisite
• Probability and Statistics, Linear Algebra, Optimization
• Schedule
• 1 week
• 1-1. Regression analysis, Simple regression model, Model estimation

[K-MOOC] Data Analytics for Forecasting and Classification: 1-1. Regression analysis, Simple regression model, Model estimation

Regression Analysis

1. In order to explain a variable, to analyze statistical causal relationships between related variables
2. independent variable: causes
3. dependent variable: outcomes

Regression Model

1. Simple Regression Model
1. 𝑿 ⇨ 𝒀
2. Observation: (𝑿₁,𝒀₁), (𝑿₂,𝒀₂), ... , (𝑿𝘯,𝒀𝘯) (𝑛 is observation number)
3. Simple Regression Model: 
1. 𝒀𝑖 = 𝜷₀ + 𝜷₁𝑿𝑖 + ℇ𝑖,    𝑖 = 1,2, ... , 𝑛
1. ℇ𝑖: error term. 
1. Assume that it follows a normal distribution with mean 0 and variance 𝛔²
2. ℇ𝑖~𝙉𝙤𝙧(0,𝛔²)
2. 𝑿 is not random variable, but a given value
3. so, three parameters need to be estimated
1. 𝜷₁: slope of the linear equation
2. 𝜷₀: intercept
3. 𝛔²: variance of the error term
4. Estimation of intercept 𝜷₀ and slope 𝜷₁
1. Using least squares method
2. to minimize the objective function 𝐐
3. objective function 𝐐
1. sum of the square of the difference between the observed value of dependent variable 𝒀, and the fitted value provided by the model on the linear line 𝜷₀ + 𝜷₁𝑿𝑖
2. 𝐐 = ∑(𝒀𝑖 - 𝜷₀ - 𝜷₁𝑿𝑖)²
5. How to?
1. (𝑿,𝒀) is observed value, so let 𝐐 be a function of 𝜷₀ and 𝜷₁ 
2. and partially differentiate 𝐐 with respect to 𝜷₀
   = -2∑(𝒀𝑖 - 𝜷₀ - 𝜷₁𝑿𝑖) = 0
3. and partially differentiate 𝐐 with respect to 𝜷₁
   = -2∑(𝒀𝑖 - 𝜷₀ - 𝜷₁𝑿𝑖)𝑿𝑖 = 0
4. estimated equation: 𝒀-hat = 𝜷₀-hat + 𝜷₁-hat * 𝑿
6. Estimation of variance of the error term 𝛔²
1. Using sample variance of the residuals
1. residual
   substract the estimated value from the observed value of 𝒀
   𝒆𝑖 = 𝒀𝑖 - 𝒀-hat = 𝒀𝑖 - 𝜷₀-hat + 𝜷₁-hat * 𝑿𝑖
2. SSE
   resudual/error sum of squares
   = ∑(𝒀𝑖 - 𝒀𝑖-hat)²
3. estimate 𝛔² by using MSE
   𝛔²-hat = MSE(Mean Squared Error) = SSE / 𝑛-2
   (𝑛-2) is  degree of freedom