[PyData] 01 - Web Crawler
前言
Let's go to https://www.kaggle.com/
Kaggle Notebook 有实践记录的案例。
一、线性拟合噪声数据
[Sklearn] Linear regression models to fit noisy data
二、打造 Pipeline
[Feature] Final pipeline: custom transformers
资源队列
阅读目录
- Algorithmic Trading Challenge25
- Allstate Purchase Prediction Challenge3
- Amazon.com – Employee Access Challenge6
- AMS 2013-2014 Solar Energy Prediction Contest2
- Belkin Energy Disaggregation Competition1
- Challenges in Representation Learning: Facial Expression Recognition Challenge4
- Challenges in Representation Learning: The Black Box Learning Challenge1
- Challenges in Representation Learning: Multi-modal Learning2
- Detecting Insults in Social Commentary
- EMI Music Data Science Hackathon
- Galaxy Zoo – The Galaxy Challenge
- Global Energy Forecasting Competition 2012 – Wind Forecasting
- KDD Cup 2013 – Author-Paper Identification Challenge (Track 1)2
- KDD Cup 2013 – Author Disambiguation Challenge (Track 2)1
- Large Scale Hierarchical Text Classification4
- Loan Default Prediction – Imperial College London
- Merck Molecular Activity Challenge1
- MLSP 2013 Bird Classification Challenge
- Observing the Dark World
- PAKDD 2014 – ASUS Malfunctional Components Prediction
- Personalize Expedia Hotel Searches – ICDM 2013
- Predicting a Biological Response1
- Predicting Closed Questions on Stack Overflow
- See Click Predict Fix1
- See Click Predict Fix – Hackathon1
- StumbleUpon Evergreen Classification Challenge
- [The Analytics Edge (15.071x)](The%20Analytics Edge (15.071x))
- The Marinexplore and Cornell University Whale Detection Challenge
- Walmart Recruiting – Store Sales Forecasting1
Kaggle比赛源代码和讨论的收集整理。
Algorithmic Trading Challenge25
Allstate Purchase Prediction Challenge3
- Rank 2 solution code21 by Alessandro Mariani.
- Rank 10 solution code5 by B1aine.
- Rank 36 solution cod1e by Hiroyuki.
- Rank 159 solution code by MrCanard.
- Solution thread.
Amazon.com – Employee Access Challenge6
- Rank 1 solution code24 by Paul Duan and Benjamin Solecki team.
- Rank 1 solution Q&A5 by Paul Duan.
- Rank 2 solution code1 by Owen Zhang.
- Rank 3 solution code3 by Dmitry & Leustagos.
- Rank 289 solution code by Foxtrot with original blog post here.
- Solution thread.
AMS 2013-2014 Solar Energy Prediction Contest2
- Rank 1 solution code5 and description4 by Leustagos team.
- Rank 2 solution code and description by Toulouse.
- Rank 3 solution code1 and description by Owen Zhang.
- Rank 4 solution escription by Peter Prettenhofer.
- Rank 5 solution description by Domcastro.
- Rank 58 solution code and description by Davit.
- Solution thread here.
- Ridge Regression starter code with MAE about 2.2M by Alec Radford, original thread here.
- Improved starter code by Foxtrot.
- Baseline code with MAE about 2.6M using Catmull-Rom Spline interpolation, also available in R here andhere.
Belkin Energy Disaggregation Competition1
Challenges in Representation Learning: Facial Expression Recognition Challenge4
- Rank 1 solution code6 and description2 by Charlie Tang.
- Rank 3 solution description3 by Maxim Milakov.
- Solution thread.
Challenges in Representation Learning: The Black Box Learning Challenge1
- Rank 1 solution description1 by David Thaler.
- Rank 2 solution code and description by sayit.
Challenges in Representation Learning: Multi-modal Learning2
- Rank 1 solution1 by MMDL.
- Solution thread.
Detecting Insults in Social Commentary
- Rank 1 solution description4 and code by Vivek Sharma.
- Rank 2 solution1 by tuzzeg.
- Rank 3 solution description Andrei Olariu.
- Rank 4 solution by Chris Brew.
- Rank 5 solution description by Yasser Tabandeh.
- Rank 6 solution by Andreas Mueller, code available here.
- Rank 8 solution description by Steve Poulson.
- Solution thread.
EMI Music Data Science Hackathon
- Rank 4 solution description1 by Steffen Rindle.
- Rank 18 solution code and description by Vlad Gusev.
- Rank 34 solution code and description by zenog.
- Solution thread.
Galaxy Zoo – The Galaxy Challenge
- Rank 1 solution code2 and description1 by Sander Dieleman.
- Rank 2 solution code and description by Maxim Milakov.
- Rank 3 solution code and description by tund.
- Rank 5 solution code and description by Julian de Wit.
- Rank 9 solution code and description by Soumith Chintala.
- Rank 13 solution code and description by Xiaoxiang Zhang.
- Rank 28 solution code and description by utdiscant.
- Rank 38 solution code and description by sugi.
- Rank 57 solution code and description1 by hxu.
- Rank 58 solution code and description by yr.
- Solution thread.
Global Energy Forecasting Competition 2012 – Wind Forecasting
- Rank 1 solution by Leustagos.
- Solution thread here1.
KDD Cup 2013 – Author-Paper Identification Challenge (Track 1)2
- Rank 1 solution with code and description4 by Team Algorithm, Github link to code here1.
KDD Cup 2013 – Author Disambiguation Challenge (Track 2)1
- Rank 1 solution with code and description4 by Team Algorithm, Github link to code here1.
- Rank 2 solution1 by SmallData Team.
- Rank 3 solution1 by hustmonk.
- Rank 4 solution1 by Ben S.
- Solution thread1.
Large Scale Hierarchical Text Classification4
- Rank 1 solution code and description7 by anttip.
- Rank 3 solution code2 and description2 by nagadomi.
- Solution thread one3.
- Solution thread two2.
Loan Default Prediction – Imperial College London
- Rank 2 solution and description1 by HelloWorld.
- Rank 12 solution and description by David McGarry.
- Solution thread.
Merck Molecular Activity Challenge1
MLSP 2013 Bird Classification Challenge
- Rank 1 solution code3 and description by beluga.
- Rank 2 solution code1 and description by Herbal Candy (W and thomeou).
- Rank 3 solution description by Anil Thomas.
- Rank 4 solution description by Maxim Milakov.
- Solution thread.
Observing the Dark World
- Rank 2 solution by Iain Murray, code available here.
PAKDD 2014 – ASUS Malfunctional Components Prediction
Personalize Expedia Hotel Searches – ICDM 2013
- Presentation paper/slides1 for ICDM 2013.
- Solution thread1.
Predicting a Biological Response1
- Rank 6 solution by Shea Parkes & Neil Schneider team.
- Rank 17 solution of Ensemble of RandomForests, GradientBoostingTrees and ExtraTreesRegressorby Emanuele Olivetti.
- Another solution code by Oblique Random Forest (oRF) by Shea Parkes & Neil Schneider team.
- The code of my best submission thread. Talks about Multi-core training Oblique Random Forests, and Stacking.
- Question about the process of ensemble learning thread. Talks about applying ensembles in practice, and how can problems arise and how to deal with them.
Predicting Closed Questions on Stack Overflow
- Rank 10 solution by Marco Lui.
- Rank 33 solution by Foxtrot.
See Click Predict Fix1
See Click Predict Fix – Hackathon1
StumbleUpon Evergreen Classification Challenge
- Benchmark beater 1.
- Benchmark beater 2.
- Benchmark beater 3.
- Solution thread.
- My own solution, which is a good example of what is overfitting. (Public rank: 57, Private rank: 291)
[The Analytics Edge (15.071x)](The%20Analytics Edge (15.071x))
- Rank 17 solution code and description by Foxtrot.
- Solution thread.
The Marinexplore and Cornell University Whale Detection Challenge
- Rank 1 solution by Nick Kridler.
- Rank 7 solution by Gilles Louppe and Peter Prettenhofer team.
- Rank 8 solution by Sander Dieleman.
- Rank 56 solution by Sudeep Juvekar.
- Solution discussion thread.
- Mean spectogram thread.
- Official interview from the Marinexplorer and Cornell at Kaggle.
Walmart Recruiting – Store Sales Forecasting1
- Rank 1 solution code5 and description by David Thaler.
- Rank 2 solution description1 by sriok.
- Rank 3 solution code and description1 by James King.
- Rank 5 solution description by ACS69.
- Rank 6 solution description by T. Henry.
- Rank 8 solution description by BreakfastPirate.
- Rank 9 solution description by Neil Summers.
- Rank 10 solution description by Gilberto Titericz Junior.
- Rank 11 solution description by citynight.
- Rank 16 solution code and description by yr.
- Rank 29 solution code and description by Mike Kim.
- Rank 30 solution description by dkay.
- Solution thread.
Thank you Foxtrot, James Petterson, Ben S for providing some of the links and solutions above.
[PyData] 01 - Web Crawler的更多相关文章
- A web crawler design for data mining
Abstract The content of the web has increasingly become a focus for academic research. Computer prog ...
- [CareerCup] 10.5 Web Crawler 网络爬虫
10.5 If you were designing a web crawler, how would you avoid getting into infinite loops? 这道题问如果让我们 ...
- (92) Web Crawling: How can I build a web crawler from scratch? - Quora
(92) Web Crawling: How can I build a web crawler from scratch? - Quora How can I build a web crawler ...
- <Web Crawler><Java><thread-safe queue>
Basic Solution The simplest way is to build a web crawler that runs on a single machine with single ...
- Free web scraping | Data extraction | Web Crawler | Octoparse, Free web scraping
Free web scraping | Data extraction | Web Crawler | Octoparse, Free web scraping 人才知了
- 01.Web大前端时代之：HTML5+CSS3入门系列~初识HTML5
Web大前端时代之:HTML5+CSS3入门系列:http://www.cnblogs.com/dunitian/p/5121725.html 文档申明 <!--文档类型申明,html代表是ht ...
- Tomcat笔记 #01# WEB应用管理工具简介
索引 查看JVM以及SERVLET/接口的情况 动态管理WEB应用 Tomcat自带了一个基于网页的web应用管理工具,可以帮助我们监控&管理部署上去的WEB APP,特别方便!恰好之前碰到的 ...
- 【Web crawler】simulated DFS web crawler
Finish crawl web learned from udacity 提示:在某些时候,你必须在page上调用get_page.这似乎违反直觉,但是我们用 page 这个词时,指的网页的网址 ( ...
- 01 Web框架介绍
一.Web框架本质 所有的web应用程序本质上都是socket,用户的浏览器其实就是一个socket客户端. python中常用的web框架有: Django Flask web.py WSGI(we ...
随机推荐
- adblock 下载地址
addblock 的作用: 防止追踪.恶意域名,过滤横幅广告.弹窗广告以及视频广告. 用以支持网站的非侵入式广告将不会被屏蔽 下载地址:https://pan.baidu.com/share/li ...
- iOS for MachineLearning
链接: 手把手教你在应用里用上iOS机器学习框架Core ML iOS11 新功能开发之 - "高大上"的 CoreML 与 Vision Core ML介绍 (Apple机器学习 ...
- 解决qt提示：qt.network.ssl: QSslSocket: cannot call unresolved function DH_free
方法一(解决):把C:\Qt\Qt5.8.0\Tools\QtCreator\bin下的libeay32.dll和ssleay32.dll库复制到C:\Qt\Qt5.8.0\5.8\msvc2015_ ...
- opencv rtsp 人脸识别
import cv2 import dlibimport jsonface_detector = dlib.get_frontal_face_detector() cap = cv2.VideoCap ...
- 修改文件夹的protection level之后，哪个job会来执行re-stripe的操作呢？
有下面的一些job可能参与其中的,他们的描述如下: AutoBalance,AutoBalanceLin - Balances free space in the cluster. The goal ...
- 这个Linux命令是干什么的？
笔者遇到一个命令,觉得挺不好懂的. find . –type f –exec dd if={} of=/dev/null bs=128K status=none \; 这个命令中: {} 代表着被找到 ...
- IE10、IE11使用 __doPostBack 出现未定义问题
在公司的老项目中分页控件使用了 __doPostBack 方式,在IE兼容模式下正常,在IE10.IE11中 __doPostBack 出现未定义问题. 百度查阅资料得知,这是微软NET环境下的一个B ...
- MySQL -- Innodb中的change buffer
change buffer是一种特殊的数据结构,当要修改的辅助索引页不在buffer pool中时,用来cache对辅助索引页的修改.对辅助索引页的操作可能是insert.update和delete操 ...
- html中<a>标签的种类
在html中a 标签是一个链接标签,然而a 标签也有非常多的种类,在此做一个小结. 一.普通链接 <a href="http://www.baidu.com">百度&l ...
- WEBAPI 的简单示例
一.webapi 1.在webapiconfig中移除xml的返回格式,返回格式就自动使用Json格式 config.Formatters.Remove(config.Formatters.XmlFo ...