scikit-笔记14:Pipeline Estimators

For some types of data, for instance text data, a feature extraction step must be applied to convert it to numerical features. To illustrate we load the SMS spam dataset we used earlier.

import os
with open(os.path.join("datasets", "smsspam", "SMSSpamCollection")) as f:
    lines = [line.strip().split("\t") for line in f.readlines()]
text = [x[1] for x in lines]
y = [x[0] == "ham" for x in lines]

from sklearn.model_selection import train_test_split
text_train, text_test, y_train, y_test = train_test_split(text, y)

Previously, we applied the feature extraction manually, like so:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
vectorizer = TfidfVectorizer()
vectorizer.fit(text_train)
X_train = vectorizer.transform(text_train)
X_test = vectorizer.transform(text_test)
clf = LogisticRegression()
clf.fit(X_train, y_train)
clf.score(X_test, y_test)

0.96700143472022959

The situation where we learn a transformation and then apply it to the test data is very common in machine learning. Therefore scikit-learn has a shortcut for this, called pipelines:

from sklearn.pipeline import make_pipeline
pipeline = make_pipeline(TfidfVectorizer(), LogisticRegression())
pipeline.fit(text_train, y_train)
pipeline.score(text_test, y_test)

0.96700143472022959

>>> what happend when we call fit/score/predict on pipeline

• When calling fit on the pipeline, it will call fit on each step in turn. ​ After the first step is fit, it will use the transform method of the first step to create a new representation. This will then be fed to the fit of the next step, and so on. Finally, on the last step, only fit is called. ​

• When calling score, only transform will be called on each step - this could be the test set after all! Then, on the last step, score is called with the new representation.
• The same goes for predict.

As you can see, this makes the code much shorter and easier to handle. Behind the scenes, exactly the same as above is happening.

Another benefit of using pipelines is that we can now also search over parameters of the feature extraction with GridSearchCV:

from sklearn.model_selection import GridSearchCV
pipeline = make_pipeline(TfidfVectorizer(), LogisticRegression())
params = {'logisticregression__C': [.1, 1, 10, 100],
          "tfidfvectorizer__ngram_range": [(1, 1), (1, 2), (2, 2)]}
grid = GridSearchCV(pipeline, param_grid=params, cv=5)
grid.fit(text_train, y_train)
print(grid.best_params_)
grid.score(text_test, y_test)

0.98565279770444758

EXERCISE: Create a pipeline out of a StandardScaler and Ridge regression and apply it to the Boston housing dataset (load using sklearn.datasets.load_boston). Try adding the sklearn.preprocessing.PolynomialFeatures transformer as a second preprocessing step, and grid-search the degree of the polynomials (try 1, 2 and 3).

1. from sklearn.datasets import make_blobs
2. from sklearn.datasets import load_iris
3. from sklearn.model_selection import train_test_split
4. from sklearn.model_selection import cross_val_score
5. from sklearn.model_selection import KFold
6. from sklearn.model_selection import StratifiedKFold
7. from sklearn.model_selection import ShuffleSplit
8. from sklearn.model_selection import GridSearchCV *
9. from sklearn.linear_model import LogisticRegression
10. from sklearn.linear_model import LinearRegression
11. from sklearn.neighbors import KNeighborsClassifier
12. from sklearn.neighbors import KNeighborsRegressor
13. from sklearn.preprocessing import StandardScaler
14. from sklearn.decomposition import PCA
15. from sklearn.metrics import confusion_matrix, accuracy_score
16. from sklearn.metrics import adjusted_rand_score
17. from sklearn.cluster import KMeans
18. from sklearn.cluster import KMeans
19. from sklearn.cluster import MeanShift
20. from sklearn.cluster import DBSCAN # <<< this algorithm has related sources in LIHONGYI's lecture-12
21. from sklearn.cluster import AffinityPropagation
22. from sklearn.cluster import SpectralClustering
23. from sklearn.cluster import Ward
24. from sklearn.metrics import confusion_matrix
25. from sklearn.metrics import accuracy_score
26. from sklearn.metrics import adjusted_rand_score
27. from sklearn.feature_extraction import DictVectorizer
28. from sklearn.feature_extraction.text import CountVectorizer
29. from sklearn.feature_extraction.text import TfidfVectorizer
30. from sklearn.preprocessing import Imputer
31. from sklearn.dummy import DummyClassifier
32. from sklearn.pipeline import make_pipeline *