Differences
This shows you the differences between two versions of the page.
| Next revision | Previous revision | ||
| blog:draft:review_data_analysis [2020/03/23 13:33] – created prgram | blog:draft:review_data_analysis [2025/07/07 14:12] (current) – external edit 127.0.0.1 | ||
|---|---|---|---|
| Line 5: | Line 5: | ||
| 클렌징 | 클렌징 | ||
| + | 다는 필요 없을 듯? | ||
| + | |||
| + | 머신러닝프로젝트 책 | ||
| + | |||
| + | https:// | ||
| + | Steve Null, Null Island | ||
| https:// | https:// | ||
| Line 19: | Line 25: | ||
| Domain Adaptation | Domain Adaptation | ||
| Noise/ | Noise/ | ||
| + | |||
| + | |||
| + | Data Properties | ||
| + | Regarding the initial dataset: | ||
| + | |||
| + | |||
| + | 어떻게 생성된 데이터인가? | ||
| + | - 샘플링 된 것인지? | ||
| + | - 예전 자료인지, | ||
| + | E.g. 10% of last month data was sampled uniformly from each of the existing five clients. | ||
| + | |||
| + | noise나 sampling bias, 결측치는 없나? | ||
| + | E.g. one of the clients only integrated with our service two week ago, introducing a down-sampling bias of his data in the dataset. | ||
| + | - 이를 줄이거나 제거하기 위해서 시도 할 수 있는 방법은? | ||
| + | E.g. either upsample the under-sampled client by a factor of two, or use data from only the last two weeks for all clients. | ||
| + | |||
| + | Can you explicitly model noise, independently from an approach? | ||
| + | |||
| + | 따로 라벨링이 된 데이터라면, | ||
| + | - 라벨링에 bias는 없나? bias를 측정할 수 있을까? | ||
| + | E.g. the label might come from semi-attentive users. Labelling a very small (but representative) set by hand using experts/ | ||
| + | - bias를 줄이기 위해서 시도할 수 있는 방법은? | ||
| + | |||
| + | 사용된 데이터가 실제 활용될 때 구조나 내용적으로 다른 점은 없나? | ||
| + | E.g. the content of some items changes dynamically in production. Or perhaps different fields are missing, depending on time of creation, or on the source domain, and are later completed or extrapolated. | ||
| + | |||
| + | 사용된 데이터가 실제를 대표할 수 있나? | ||
| + | E.g. The distribution of data among clients constantly changes. Or perhaps it was sampled over two month of spring but the model will go up when winter starts. Or it might have been collected before a major client/ | ||
| + | |||
| + | What is the best training dataset you could hope for? | ||
| + | - Define it very explicitly. | ||
| + | - Estimate: By how much will it improve performance? | ||
| + | - How possible and costly is it to generate? | ||
| + | E.g. tag the sentiment of 20,000 posts using three annotators and give the mode/ | ||