一、前言

在实际项目中，数据分析往往不是单一步骤，而是一个完整的全流程：

1. 数据获取与导入

2. 数据清洗与预处理

3. 特征工程

4. 数据建模

5. 模型调优

6. 数据可视化与报告

本文将以 Titanic 数据集为例，展示 Python 数据分析的全流程实战。

---

二、数据导入与初步探索

import pandas as pd import seaborn as sns df = sns.load_dataset('titanic') df.head() df.info() df.describe()

• 查看数据结构、字段类型、缺失值情况

• 确定目标变量和特征变量

---

三、数据清洗与预处理

1. 缺失值处理

df['age'] = df['age'].fillna(df['age'].median()) df['embarked'] = df['embarked'].fillna(df['embarked'].mode()[0])

1. 重复值处理

df = df.drop_duplicates()

1. 类别变量编码

df['sex'] = df['sex'].map({'male':0, 'female':1}) df = pd.get_dummies(df, columns=['embarked'], drop_first=True)

---

四、特征工程

1. 标准化数值变量

from sklearn.preprocessing import StandardScaler scaler = StandardScaler() df[['age','fare']] = scaler.fit_transform(df[['age','fare']])

1. 生成新特征

df['age_fare_ratio'] = df['age'] / (df['fare'] + 1)

1. 选择特征

X = df.drop('survived', axis=1) y = df['survived']

---

五、数据建模与训练

1. 划分训练集和测试集

from sklearn.model_selection import train_test_split X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

1. 逻辑回归分类模型

from sklearn.linear_model import LogisticRegression from sklearn.metrics import accuracy_score, classification_report clf = LogisticRegression(max_iter=200) clf.fit(X_train, y_train) y_pred = clf.predict(X_test) print("Accuracy:", accuracy_score(y_test, y_pred)) print(classification_report(y_test, y_pred))

1. 随机森林模型

from sklearn.ensemble import RandomForestClassifier rf_clf = RandomForestClassifier(n_estimators=100, max_depth=5, random_state=42) rf_clf.fit(X_train, y_train) y_pred_rf = rf_clf.predict(X_test) print("Random Forest Accuracy:", accuracy_score(y_test, y_pred_rf))

---

六、模型调优

from sklearn.model_selection import GridSearchCV param_grid = {'n_estimators':[50,100,200], 'max_depth':[3,5,7]} grid_search = GridSearchCV(rf_clf, param_grid, cv=5, scoring='accuracy') grid_search.fit(X_train, y_train) print("Best Parameters:", grid_search.best_params_) print("Best Score:", grid_search.best_score_)

• 网格搜索 + 交叉验证找到最佳参数，提高模型泛化能力

---

七、数据可视化与洞察

1. 类别分布

import matplotlib.pyplot as plt import seaborn as sns sns.countplot(x='survived', hue='sex', data=df) plt.title('Survival Count by Gender') plt.show()

1. 特征与目标关系

sns.boxplot(x='pclass', y='age', data=df) plt.title('Age Distribution by Class') plt.show()

1. 相关性热力图

corr = df.corr() sns.heatmap(corr, annot=True, cmap='coolwarm') plt.title('Correlation Heatmap') plt.show()

• 可视化帮助发现特征与目标的关系，指导建模和决策