scikit-笔记04:监督学习之分类问题

First, we will look at a two class classification problem in two dimensions. We use the synthetic data generated by the make_blobs function.

sklearn.datasets.make_blobs

# definition of make_blobs: produce multiple gaussians
sklearn.datasets.make_blobs(n_samples=100,
                            n_features=2,
                            centers=3,
                            cluster_std=1.0,
                            center_box=(-10.0, 10.0),
                            shuffle=True,
                            random_state=None)

from sklearn.datasets import make_blobs

X, y = make_blobs(centers=2, random_state=0)

print('X ~ n_samples x n_features:', X.shape)
print('y ~ n_samples:', y.shape)

print('\nFirst 5 samples:\n', X[:5, :])
print('\nFirst 5 labels:', y[:5])

As the data is two-dimensional, we can plot each sample as a point in a two-dimensional coordinate system, with the first feature being the x-axis and the second feature being the y-axis.

import matplotlib.pyplot as plt
import numpy as np

# give different color to data with different label
plt.scatter(X[y == 0, 0], X[y == 0, 1],
            c='blue', s=40, label='0')
plt.scatter(X[y == 1, 0], X[y == 1, 1],
            c='red', s=40, label='1', marker='s')

plt.xlabel('first feature')
plt.ylabel('second feature')
plt.legend(loc='upper right')

<matplotlib.legend.Legend at 0x7f9c530a16d8>

Classification is a supervised task, and since we are interested in its performance on unseen data, we split our data into two parts:

• a training set that the learning algorithm uses to fit the model
• a test set to evaluate the generalization performance of the model

The train_test_split function from the model_selection module does that for us – we will use it to split a dataset into 75% training data and 25% test data.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(X, y,
                                                    test_size=0.25,
                                                    random_state=1234,
                                                    stratify=y)

# stratify : array-like or None (default is None)
# If not None, data is split in a stratified fashion, using this as the class labels.

Estimator is an ML algorithm, like linear/logistic regression, SVM etc.

from sklearn.linear_model import LogisticRegression

classifier = LogisticRegression()
X_train.shape
y_train.shape

(75,)

estimator.fit(train_data, train_label) will print the initial parameters of this model.

To built the model from our data, that is to learn how to classify new points, we call the fit function with the training data, and the corresponding training labels (the desired output for the training data point):

classifier.fit(X_train, y_train)

LogisticRegression(C=1.0, class_weight=None, dual=False, fit_intercept=True,
intercept_scaling=1, max_iter=100, multi_class='ovr', n_jobs=1,
penalty='l2', random_state=None, solver='liblinear', tol=0.0001,
verbose=0, warm_start=False)

(Some estimator methods such as fit return self by default. Thus, after executing the code snippet above, you will see the default parameters of this particular instance of LogisticRegression. Another way of retrieving the estimator's ininitialization parameters is to execute classifier.get_params(), which returns a parameter dictionary.)

estimator.predict(test_data) will return the predict_test_label as ndarray.

We can then apply the model to unseen data and use the model to predict the estimated outcome using the predict method:

prediction = classifier.predict(X_test)
prediction

array([1, 0, 1, 0, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 1, 0, 0, 1, 0, 0, 0, 1,
1, 0])

We can compare these against the true labels:

print(prediction)
print(y_test)

We can evaluate our classifier quantitatively by measuring what fraction of predictions is correct. This is called accuracy:

np.mean(prediction == y_test) # <- classical usage of mean, keep in mind

0.83999999999999997

There is also a convenience function , score, that all scikit-learn classifiers have to compute this directly from the test data:

classifier.score(X_test, y_test)

0.83999999999999997

It is often helpful to compare the generalization performance (on the test set) to the performance on the training set:

classifier.score(X_train, y_train)

0.94666666666666666

LogisticRegression is a so-called linear model, that means it will create a decision that is linear in the input space. In 2d, this simply means it finds a line to separate the blue from the red:

plot_2d_separator is a method defined in ./figures/plot_2d_separator.py

from figures import plot_2d_separator

plt.scatter(X[y == 0, 0], X[y == 0, 1],
            c='blue', s=40, label='0')
plt.scatter(X[y == 1, 0], X[y == 1, 1],
            c='red', s=40, label='1', marker='s')

plt.xlabel("first feature")
plt.ylabel("second feature")
plot_2d_separator(classifier, X)
plt.legend(loc='upper right');

Estimated parameters: All the estimated model parameters are attributes of the estimator object ending by an underscore. Here, these are the coefficients and the offset of the line:

print(classifier.coef_)
print(classifier.intercept_)

Generate isotropic Gaussian blobs for clustering.

n_samples : int, optional (default=100) The total number of points equally divided among clusters.

n_features : int, optional (default=2) The number of features for each sample.

centers : int or array of shape [n_centers, n_features], optional (default=3) The number of centers to generate, or the fixed center locations.

cluster_std : float or sequence of floats, optional (default=1.0) The standard deviation of the clusters.

center_box : pair of floats (min, max), optional (default=(-10.0, 10.0)) The bounding box for each cluster center when centers are generated at random.

shuffle : boolean, optional (default=True) Shuffle the samples.

random_state : int, RandomState instance or None, optional (default=None)

sklearn.datasets.make_blobs(n_samples=100,
                            n_features=2,
                            centers=3,
                            cluster_std=1.0,
                            center_box=(-10.0, 10.0),
                            shuffle=True,
                            random_state=None)

1. from sklearn.neighbors import KNeighborsClassifier
2. from sklearn.datasets import make_blobs
3. from sklearn.model_selection import train_test_split
4. from sklearn.linear_model import LogisticRegression

. | | X_col0 | X_col1 | | | X_col1 | . |----------------------------–—| |---–—+-------–—| . | X_row0 | 4.21850347 | 2.23419161 | . | X_row1 | 0.90779887 | 0.45984362 | X[ mask, 1] . | X_row2 | -0.27652528 | 5.08127768 | keep the true row and col 1 | X_row2 | 5.08127768 | . | X_row3 | 0.08848433 | 2.32299086 | ^ | X_row3 | 2.32299086 | . | X_row4 | 3.24329731 | 1.21460627 | | . | . -------- . | . mask = boolean of array . | y | | y | . |—| |—| . | 1 | | F | . | 1 | y == 0 | F | . | 0 | -------------------------–—> | T | . | 0 | | T | . | 1 | | F |

from sklearn.datasets import make_blobs
import matplotlib.pyplot as plt
import numpy as np

X, y = make_blobs(centers=2, random_state=0)

print('X ~ n_samples x n_features:', X.shape)
print('y ~ n_samples:', y.shape)

print('\nFirst 5 samples:\n', X[:5, :])
print('\nFirst 5 labels:', y[:5])

plt.scatter(X[y == 0, 0], X[y == 0, 1],
            c='blue', s=40, label='0')
plt.scatter(X[y == 1, 0], X[y == 1, 1],
            c='red', s=40, label='1', marker='s')

plt.xlabel('first feature')
plt.ylabel('second feature')
plt.legend(loc='upper right')

. sklearn.datasets np pd pd . make_blobs -----–—> ndarray ----–—> dict -----–—> dataframe ----–—> groupby . and color dict

. for i,j in groupOfDf: plot (df,color=color_dict[i])

from sklearn.datasets.samples_generator import make_blobs
from matplotlib import pyplot
from pandas import DataFrame
# generate 2d classification dataset
X, y = make_blobs(n_samples=100, centers=3, n_features=2)
# scatter plot, dots colored by class value
df = DataFrame(dict(x=X[:,0], y=X[:,1], label=y))
colors = {0:'red', 1:'blue', 2:'green'}
fig, ax = pyplot.subplots()
grouped = df.groupby('label')
for key, group in grouped:
    group.plot(ax=ax, kind='scatter', x='x', y='y', label=key, color=colors[key])
pyplot.show()