请注意，文章中部分理解可能已被笔者废弃，本文最后更新于：1 个月前
因子分析（Factor Analysis，FA）
选择分析变量
计算所选原始变量的相关系数矩阵
提取公共因子，确定因子数目
因子旋转，解释因子
计算因子得分
本文的变量名大体可分为三种：大写粗体、大写、小写。n n n
记 m m m X i ( i = 1 , 2 , … , m ) X_i(i=1,2,\dots,m) X i ( i = 1 , 2 , … , m )
X i = μ i + α i 1 F 1 + ⋯ + α i m F m + ε i X_{i}=\mu_{i}+\alpha_{i 1} F_{1}+\cdots+\alpha_{i m} F_{m}+\varepsilon_{i}
X i = μ i + α i 1 F 1 + ⋯ + α i m F m + ε i
即
[ X 1 X 2 ⋮ X m ] = [ μ 1 μ 2 ⋮ μ m ] + [ α 11 α 12 ⋯ α 1 r α 21 α 22 ⋯ α 2 r ⋮ ⋮ ⋮ α m 1 α m 2 ⋯ α m r ] [ F 1 F 2 ⋮ F r ] + [ ε 1 ε 2 ⋮ ε m ] \left[\begin{array}{c}
X_{1} \\
X_{2} \\
\vdots \\
X_{m}
\end{array}\right]=\left[\begin{array}{c}
\mu_{1} \\
\mu_{2} \\
\vdots \\
\mu_{m}
\end{array}\right]+\left[\begin{array}{cccc}
\alpha_{11} & \alpha_{12} & \cdots & \alpha_{1 r} \\
\alpha_{21} & \alpha_{22} & \cdots & \alpha_{2 r} \\
\vdots & \vdots & & \vdots \\
\alpha_{m 1} & \alpha_{m 2} & \cdots & \alpha_{m r}
\end{array}\right]\left[\begin{array}{c}
F_{1} \\
F_{2} \\
\vdots \\
F_{r}
\end{array}\right]+\left[\begin{array}{c}
\varepsilon_{1} \\
\varepsilon_{2} \\
\vdots\\
\varepsilon_{m}
\end{array}\right] ⎣ ⎢ ⎢ ⎢ ⎡ X 1 X 2 ⋮ X m ⎦ ⎥ ⎥ ⎥ ⎤ = ⎣ ⎢ ⎢ ⎢ ⎡ μ 1 μ 2 ⋮ μ m ⎦ ⎥ ⎥ ⎥ ⎤ + ⎣ ⎢ ⎢ ⎢ ⎡ α 1 1 α 2 1 ⋮ α m 1 α 1 2 α 2 2 ⋮ α m 2 ⋯ ⋯ ⋯ α 1 r α 2 r ⋮ α m r ⎦ ⎥ ⎥ ⎥ ⎤ ⎣ ⎢ ⎢ ⎢ ⎡ F 1 F 2 ⋮ F r ⎦ ⎥ ⎥ ⎥ ⎤ + ⎣ ⎢ ⎢ ⎢ ⎡ ε 1 ε 2 ⋮ ε m ⎦ ⎥ ⎥ ⎥ ⎤
即
X − μ = Λ F + ε \boldsymbol{X}-\boldsymbol{\mu}=\boldsymbol{\Lambda}\boldsymbol{F}+\boldsymbol{\varepsilon}
X − μ = Λ F + ε
满足：
E ( F ) = 0 E(\boldsymbol{F})=0 E ( F ) = 0 E ( ε ) = 0 E(\boldsymbol{\varepsilon})=0 E ( ε ) = 0 载荷因子 F i F_i F i C o v ( F ) = E m Cov(\boldsymbol{F})=E_m C o v ( F ) = E m
特殊因子 ε i \varepsilon_i ε i C o v ( ε ) = D ( ε ) = d i a g ( σ 1 2 , σ 2 2 , … , σ m 2 ) Cov(\boldsymbol{\varepsilon})=D(\boldsymbol{\varepsilon})=diag(\sigma _1^2,\sigma _2^2,\dots,\sigma _m^2) C o v ( ε ) = D ( ε ) = d i a g ( σ 1 2 , σ 2 2 , … , σ m 2 )
特殊因子和载荷因子无线性关系：C o v ( F , ε ) = 0 Cov(\boldsymbol{F},\boldsymbol{\varepsilon})=0 C o v ( F , ε ) = 0
原子变量 X \boldsymbol{X} X
由 X − μ = Λ F + ε \boldsymbol{X}-\boldsymbol{\mu}=\boldsymbol{\Lambda}\boldsymbol{F}+\boldsymbol{\varepsilon} X − μ = Λ F + ε C o v ( X − μ ) = Λ C o v ( F ) Λ T + C o v ( ε ) Cov(\boldsymbol{X}-\boldsymbol{\mu})=\boldsymbol{\Lambda}Cov(\boldsymbol{F})\boldsymbol{\Lambda}^T+Cov(\boldsymbol{\varepsilon}) C o v ( X − μ ) = Λ C o v ( F ) Λ T + C o v ( ε )
C o v ( X ) = Λ Λ T + D ( Λ ) Cov(\boldsymbol{X})=\boldsymbol{\Lambda}\boldsymbol{\Lambda}^T+D(\boldsymbol{\Lambda})
C o v ( X ) = Λ Λ T + D ( Λ )
σ 1 2 , σ 2 2 , … , σ m 2 \sigma _1^2,\sigma _2^2,\dots,\sigma _m^2 σ 1 2 , σ 2 2 , … , σ m 2
设 Γ \Gamma Γ
Λ ~ = Λ Γ , F ~ = Γ T F \widetilde{\boldsymbol{\Lambda}}=\boldsymbol{\Lambda}\Gamma , \widetilde{\boldsymbol{F}}=\Gamma^{\tiny T}\boldsymbol{F}
Λ = Λ Γ , F = Γ T F
X = μ + Λ ~ F ~ + ε \boldsymbol{X}=\boldsymbol{\mu}+\widetilde{\boldsymbol{\Lambda}}\widetilde{\boldsymbol{F}}+\boldsymbol{\varepsilon}
X = μ + Λ F + ε
因子载荷矩阵 α i j \alpha_{i j} α i j i i i j j j i i i j j j ∣ α i j ∣ |\alpha_{i j}| ∣ α i j ∣
变量 X i X_i X i i i i h i 2 = ∑ j = 1 r α i j 2 h_i^2=\sum\limits_{j=1}^{r}\alpha_{ij}^2 h i 2 = j = 1 ∑ r α i j 2
因为
V a r ( X i ) = α i 1 2 V a r ( F 1 ) + ⋯ + α i r 2 V a r ( F r ) + V a r ( ε i ) Var(X_i)=\alpha_{i1}^2Var(F_1)+\dots+\alpha_{ir}^2Var(F_r)+Var(\varepsilon_i)
V a r ( X i ) = α i 1 2 V a r ( F 1 ) + ⋯ + α i r 2 V a r ( F r ) + V a r ( ε i )
得
1 = ∑ j = 1 r α i j 2 + σ i 2 1=\sum\limits_{j=1}^{r}\alpha_{ij}^2+\sigma_i^2
1 = j = 1 ∑ r α i j 2 + σ i 2
可以看出 h i 2 h_i^2 h i 2 1 1 1 σ i 2 \sigma_i^2 σ i 2 0 0 0 X i X_i X i
因子 F j F_j F j j j j S j = ∑ i = 1 m α i j 2 = λ j S_j=\sum\limits_{i=1}^{m} \alpha_{ij}^2=\lambda_j S j = i = 1 ∑ m α i j 2 = λ j
用于衡量 F j F_j F j
用原协方差阵减去公因子协方差阵与特殊因子协方差阵，得到残差阵
( ϵ i j ) m × m = R − ( Λ Λ T + D ) (\epsilon_{ij})_{m\times m} = \boldsymbol{R}-(\boldsymbol{\Lambda}\boldsymbol{\Lambda}^T+\boldsymbol{D})
( ϵ i j ) m × m = R − ( Λ Λ T + D )
残差阵元素的平方和为残差平方和
Q ( m ) = ∑ i = 1 m ∑ j = 1 m ϵ i j 2 ≤ ∑ k = r + 1 m λ k 2 Q(m)=\sum\limits_{i=1}^{m}\sum\limits_{j=1}^{m}\epsilon_{ij}^2 \le \sum\limits_{k=r+1}^{m}\lambda_k^2
Q ( m ) = i = 1 ∑ m j = 1 ∑ m ϵ i j 2 ≤ k = r + 1 ∑ m λ k 2
和“主成分分析”本质相同，主成分分析是将样本经过载荷矩阵 P \boldsymbol{P} P R r \R^r R r Λ \boldsymbol{\Lambda} Λ
设样本相关系数矩阵为 R \boldsymbol{R} R λ 1 , λ 2 , … , λ m \lambda_1,\lambda_2,\dots,\lambda_m λ 1 , λ 2 , … , λ m λ 1 ≥ λ 2 ≥ ⋯ ≥ λ m \lambda_1\ge \lambda_2\ge \dots\ge \lambda_m λ 1 ≥ λ 2 ≥ ⋯ ≥ λ m η 1 , η 2 , … , η m \boldsymbol{\eta} _1,\boldsymbol{\eta} _2,\dots,\boldsymbol{\eta} _m η 1 , η 2 , … , η m H = [ η 1 , η 2 , … , η m ] \boldsymbol{H}=[\boldsymbol{\eta }_1,\boldsymbol{\eta }_2,\dots,\boldsymbol{\eta }_m] H = [ η 1 , η 2 , … , η m ]
R = H [ λ 1 λ 2 ⋱ λ 1 ] H T = Λ Λ T + D \boldsymbol{R}=\boldsymbol{H}\begin{bmatrix}
\boldsymbol{\lambda_1}& & & \\
& \boldsymbol{\lambda_2} & & \\
& &\ddots & \\
& & & \boldsymbol{\lambda_1}
\end{bmatrix}\boldsymbol{H}^T=\boldsymbol{\Lambda}\boldsymbol{\Lambda}^T+D R = H ⎣ ⎢ ⎢ ⎡ λ 1 λ 2 ⋱ λ 1 ⎦ ⎥ ⎥ ⎤ H T = Λ Λ T + D
（ D \small\boldsymbol{D} D
Λ = [ λ 1 η 1 , λ 2 η 2 , … , λ m η m ] \boldsymbol{\Lambda}=[\sqrt{\lambda_1}\boldsymbol{\eta_1},\sqrt{\lambda_2}\boldsymbol{\eta_2},\dots,\sqrt{\lambda_m}\boldsymbol{\eta_m}]
Λ = [ λ 1 η 1 , λ 2 η 2 , … , λ m η m ]
使得公共因子方差贡献S j = λ j S_j=\lambda_j S j = λ j
根据碎石图，用碎石原则确定因子个数 r r r
特殊因子方差可以用 R − Λ Λ T \boldsymbol{R}-\boldsymbol{\Lambda}\boldsymbol{\Lambda}^T R − Λ Λ T
σ i 2 = 1 − ∑ j = 1 m α i j 2 \sigma_i^2=1-\sum_{j=1}^m\alpha_{ij}^2
σ i 2 = 1 − j = 1 ∑ m α i j 2
缺点 ：上式有一个假定，模型中的特殊因子是不重要的，因而从 R \boldsymbol{R} R 分解中忽略了特殊因子的方差 。所得的特殊因子 ε 1 ， ε 2 , . . . , ε p \varepsilon_{1}，\varepsilon_{2},...,\varepsilon_{p} ε 1 ， ε 2 , . . . , ε p
主因子方法是对主成分方法的修正，假定我们首先对变量进行标准化变换。则
R = Λ Λ T + D R ∗ = Λ Λ T = R − D \boldsymbol{R}=\boldsymbol{\Lambda}\boldsymbol{\Lambda}^T+\boldsymbol{D}\\
\boldsymbol{R}^*=\boldsymbol{\Lambda}\boldsymbol{\Lambda}^T=\boldsymbol{R}-\boldsymbol{D} R = Λ Λ T + D R ∗ = Λ Λ T = R − D
式中 R ∗ \boldsymbol{R}^* R ∗ h i 2 h_i^2 h i 2
特殊因子方差是未知的，一般通过样本估计得到。方法可以有；
取 h i 2 ^ = 1 \hat{h_i^2}=1 h i 2 ^ = 1
取 h i 2 ^ = max i ≠ j ∣ r i j ∣ \hat{h_i^2}=\max\limits_{\tiny i \ne j}|r_{ij}| h i 2 ^ = i = j max ∣ r i j ∣ X i X_i X i
取 h i 2 ^ = 1 m − 1 ∑ s j = 1 , j ≠ i m r i j \hat{h_i^2}=\frac{1}{m-1}\sum\limits_{sj=1,j\ne i}^{m}r_{ij} h i 2 ^ = m − 1 1 s j = 1 , j = i ∑ m r i j
记
R ∗ = R − D = [ h 1 2 ^ r 12 ⋯ r 1 m r 21 h 2 2 ^ ⋯ r 2 m ⋮ ⋮ ⋮ r m 1 r m 2 ⋯ h m 2 ^ ] \boldsymbol{R}^*=\boldsymbol{R}-\boldsymbol{D}=\left[\begin{array}{cccc}
\hat{h_{1}^{2}} & r_{12} & \cdots & r_{1 m} \\
r_{21} & \hat{h_{2}^{2}} & \cdots & r_{2 m} \\
\vdots & \vdots & & \vdots \\
r_{m 1} & r_{m 2} & \cdots & \hat{h_{m}^{2}}
\end{array}\right] R ∗ = R − D = ⎣ ⎢ ⎢ ⎢ ⎢ ⎡ h 1 2 ^ r 2 1 ⋮ r m 1 r 1 2 h 2 2 ^ ⋮ r m 2 ⋯ ⋯ ⋯ r 1 m r 2 m ⋮ h m 2 ^ ⎦ ⎥ ⎥ ⎥ ⎥ ⎤
直接求出 R ∗ \boldsymbol{R}^* R ∗ λ 1 ∗ , λ 2 ∗ , … , λ m ∗ \lambda_1^*,\lambda_2^*,\dots,\lambda_m^* λ 1 ∗ , λ 2 ∗ , … , λ m ∗ λ 1 ∗ ≥ λ 2 ∗ ≥ ⋯ ≥ λ m ∗ \lambda_1^*\ge \lambda_2^*\ge \dots\ge \lambda_m^* λ 1 ∗ ≥ λ 2 ∗ ≥ ⋯ ≥ λ m ∗ η 1 ∗ , η 2 ∗ , … , η m ∗ \boldsymbol{\eta} _1^*,\boldsymbol{\eta} _2^*,\dots,\boldsymbol{\eta} _m^* η 1 ∗ , η 2 ∗ , … , η m ∗
Λ = [ λ 1 ∗ η 1 ∗ , λ 2 ∗ η 2 ∗ , … , λ m ∗ η m ∗ ] \boldsymbol{\Lambda}=[\sqrt{\lambda_1^*}\boldsymbol{\eta}_1^*,\sqrt{\lambda_2^*}\boldsymbol{\eta}_2^*,\dots,\sqrt{\lambda_m^*}\boldsymbol{\eta}_m^*]
Λ = [ λ 1 ∗ η 1 ∗ , λ 2 ∗ η 2 ∗ , … , λ m ∗ η m ∗ ]
假设数据 X 1 , . . . , X n X_1,...,X_n X 1 , . . . , X n m m m
L ( μ , Λ , D ) = ∏ i = 1 d 1 ( 2 π ) m / 2 ∣ R ∣ 1 / 2 exp [ − 1 2 ( X i − μ ) ′ R − 1 ( X i − μ ) ] L(\mu, \boldsymbol{\Lambda}, \boldsymbol{D})=\prod_{i=1}^{d} \frac{1}{(2 \pi)^{m / 2}|\boldsymbol{R}|^{1 / 2}} \exp \left[-\frac{1}{2}\left(\boldsymbol{X}_{\mathbf{i}}-\mu\right)^{\prime} \boldsymbol{\boldsymbol{R}}^{-1}\left(\boldsymbol{X}_{\mathbf{i}}-\mu\right)\right]
L ( μ , Λ , D ) = i = 1 ∏ d ( 2 π ) m / 2 ∣ R ∣ 1 / 2 1 exp [ − 2 1 ( X i − μ ) ′ R − 1 ( X i − μ ) ]
用数值极大化的方法可以得到极大似然估计。
Matlab 工具箱求因子载荷矩阵使用的是最大似然估计，其命令为 factoran 。
建立因子模型不仅要得到公共因子，还要能解释这些公共因子的具体含义。
由于因子载荷矩阵不唯一，所以可以对载荷矩阵进行旋转，使得载荷矩阵每行或每列的元素平方值向 0 0 0 1 1 1
X = Λ Γ Γ − 1 F + ε \boldsymbol{X} = \boldsymbol{\Lambda }\Gamma {\Gamma ^{ - 1}} \boldsymbol{F} + \boldsymbol{\varepsilon}
X = Λ Γ Γ − 1 F + ε
因为是对整个 R r \R^r R r 变量共同度 h i 2 h_i^2 h i 2 ，因子方差贡献 S j S_j S j （正交旋转不改变向量长度）。
对于任意两列，正交矩阵形式可以设为
Γ = [ cos ϕ − sin ϕ sin ϕ cos ϕ ] \Gamma=\left[\begin{array}{ll} \cos \phi & -\sin \phi \\ \sin \phi & \cos \phi \end{array}\right]\\
Γ = [ cos ϕ sin ϕ − sin ϕ cos ϕ ]
带入得到
Λ ∗ = Λ Γ = [ α 11 cos ϕ + α 12 sin ϕ − α 11 sin ϕ + α 12 cos ϕ ⋮ ⋮ α p 1 cos ϕ + a p 2 sin ϕ − α p 1 sin ϕ + α p 2 cos ϕ ] = [ α 11 ∗ α 12 ∗ ⋮ ⋮ α m 1 ∗ α m 2 ∗ ] \boldsymbol{\Lambda}^*=\boldsymbol{\Lambda} \Gamma=\left[\begin{array}{ll} \alpha_{11} \cos \phi+\alpha_{12} \sin \phi & -\alpha_{11} \sin \phi+\alpha_{12} \cos \phi \\ \vdots & \vdots \\ \alpha_{p 1} \cos \phi+a_{p 2} \sin \phi & -\alpha_{p 1} \sin \phi+\alpha_{p 2} \cos \phi \end{array}\right]=
\left[\begin{array}{ll}
\alpha_{11}^* & \alpha_{12}^*\\
\vdots & \vdots \\
\alpha_{m 1}^* &\alpha_{m2}^*
\end{array}\right] Λ ∗ = Λ Γ = ⎣ ⎢ ⎡ α 1 1 cos ϕ + α 1 2 sin ϕ ⋮ α p 1 cos ϕ + a p 2 sin ϕ − α 1 1 sin ϕ + α 1 2 cos ϕ ⋮ − α p 1 sin ϕ + α p 2 cos ϕ ⎦ ⎥ ⎤ = ⎣ ⎢ ⎡ α 1 1 ∗ ⋮ α m 1 ∗ α 1 2 ∗ ⋮ α m 2 ∗ ⎦ ⎥ ⎤
基本思想： α i j ∗ \alpha_{ij}^* α i j ∗ i i i j j j 使因子载荷矩阵的每一列元素的方差更大 。
因为 r < m r<m r < m h i 2 h_i^2 h i 2
d i j = α i j ∗ 2 h i 2 , ( i = 1 , 2 , … , m , j = 1 , 2 , … , r ) d_{i j}=\frac{ { \alpha_{i j}^{*} }^2 }{h_i^2} , \scriptsize (i=1,2,\dots,m,j=1,2,\dots,r)
d i j = h i 2 α i j ∗ 2 , ( i = 1 , 2 , … , m , j = 1 , 2 , … , r )
定义因子载荷第 j 列的方差为
V j = 1 m ∑ i = 1 m ( d i j 2 − d ˉ j ) 2 = 1 m 2 [ m ∑ i = 1 m d i j 2 − ( ∑ i = 1 m d i j ) 2 ] V_{j}=\frac{1}{m}\sum\limits_{\tiny i=1}^{\tiny m}\left(d_{i j}^{2}-\bar{d}_{j}\right)^{2}=\frac{1}{m^2}\left[m \sum_{i=1}^{m} d_{ij}^2-{\left(\sum_{i=1}^{m} d_{ij}\right)}^{2}\right]\\
V j = m 1 i = 1 ∑ m ( d i j 2 − d ˉ j ) 2 = m 2 1 ⎣ ⎡ m i = 1 ∑ m d i j 2 − ( i = 1 ∑ m d i j ) 2 ⎦ ⎤
其中 d ˉ j = 1 m ∑ i = 1 m d i j 2 \bar{d}_{j}=\frac{1}{m} \sum\limits_{\tiny i=1}^{\tiny m} d_{i j}^{2} d ˉ j = m 1 i = 1 ∑ m d i j 2
定义因子载荷矩阵 Λ \boldsymbol{\Lambda} Λ
V = ∑ j = 1 r V j = 1 m 2 ∑ j = 1 r [ m ∑ i = 1 m d i j 2 − ( ∑ i = 1 m d i j ) 2 ] V=\sum_{j=1}^rV_{j}=\frac{1}{m^2}\sum_{j=1}^{r}\left[m \sum_{i=1}^{m} d_{ij}^2-{\left(\sum_{i=1}^{m} d_{ij}\right)}^{2}\right]
V = j = 1 ∑ r V j = m 2 1 j = 1 ∑ r ⎣ ⎡ m i = 1 ∑ m d i j 2 − ( i = 1 ∑ m d i j ) 2 ⎦ ⎤
设因子载荷矩阵有两列 r = 2 r=2 r = 2
Λ ∗ = Λ Γ = [ α 11 ∗ α 12 ∗ ⋮ ⋮ α m 1 ∗ α m 2 ∗ ] \boldsymbol{\Lambda}^*=\boldsymbol{\Lambda} \Gamma=\left[\begin{array}{ll}
\alpha_{11}^* & \alpha_{12}^*\\
\vdots & \vdots \\
\alpha_{m 1}^* &\alpha_{m2}^*
\end{array}\right]\\ Λ ∗ = Λ Γ = ⎣ ⎢ ⎡ α 1 1 ∗ ⋮ α m 1 ∗ α 1 2 ∗ ⋮ α m 2 ∗ ⎦ ⎥ ⎤
将 Λ ∗ \boldsymbol{\Lambda}^* Λ ∗ V Λ V_{\boldsymbol{\Lambda}} V Λ V Λ ∗ V_{\boldsymbol{\Lambda}^*} V Λ ∗ ϕ \phi ϕ
令
∂ V Λ ∂ ϕ = 0 \frac{\partial V_{\boldsymbol{\Lambda}}}{\partial \phi}=0
∂ ϕ ∂ V Λ = 0
得到 ϕ \phi ϕ
tan 4 ϕ = d − 2 a b / m c − ( a 2 − b 2 ) / m \tan 4 \phi=\frac{d-2 ab / m}{c-\left(a^{2}-b^{2}\right) / m}
tan 4 ϕ = c − ( a 2 − b 2 ) / m d − 2 a b / m
其中，若记
μ i = d i 1 − d i 2 , ν i = 2 α i 1 α i 2 h i 2 = 2 d i 1 d i 2 \mu_{i}=d_{i1}-d_{i2} , \quad \nu_{i}=2 \frac{\alpha_{i 1}\alpha_{i 2}}{h_{i}^{2}}=2\sqrt{d_{i1}d_{i2}}
μ i = d i 1 − d i 2 , ν i = 2 h i 2 α i 1 α i 2 = 2 d i 1 d i 2
则
a = ∑ i = 1 m μ i , b = ∑ i = 1 m ν i , c = ∑ i = 1 m ( μ i 2 − ν i 2 ) , d = 2 ∑ i = 1 m μ i ν i . \begin{array}{c}
a=\sum\limits_{i=1}^{m} \mu_{i}, \quad b=\sum\limits_{i=1}^{m} \nu_{i}, \\
c=\sum\limits_{i=1}^{m}\left(\mu_{i}^{2}-\nu_{i}^{2}\right), \quad d=2 \sum\limits_{i=1}^{m} \mu_{i} \nu_{i} .
\end{array}
a = i = 1 ∑ m μ i , b = i = 1 ∑ m ν i , c = i = 1 ∑ m ( μ i 2 − ν i 2 ) , d = 2 i = 1 ∑ m μ i ν i .
当 m > 2 m{\small>}2 m > 2 F i , F j ( i ≠ j ) F_{i}, F_{j}\small (i \neq j) F i , F j ( i = j )
选择正交旋转的角度 φ i j \varphi_{i j} φ i j m m m C m 2 C_{m}^{2} C m 2 V ( 1 ) V_{(1)} V ( 1 ) V ( 1 ) V_{(1)} V ( 1 ) V V V
基本思想： 使得因子载荷的每一行只在少部分地方取较大的值，每个变量只在一个因子上有较高的载荷，即 使因子载荷矩阵每一行的元素方差最大化 。
用于度量方差的量为
Q = ∑ i = 1 m ∑ j = 1 r ( α i j ∗ 2 − 1 r ) 2 = 化 ∑ i = 1 m ∑ j = 1 r ( α i j ∗ 4 − 2 + m r ) Q=\sum_{i=1}^{m} \sum_{j=1}^{r}\left({\alpha_{i j}^{*}}^2-\frac{1}{r}\right)^{2} \overset{\tiny 化}{=} \sum_{i=1}^{m} \sum_{j=1}^{r}\left({\alpha_{i j}^{*}}^4-2+\frac{m}{r}\right)
Q = i = 1 ∑ m j = 1 ∑ r ( α i j ∗ 2 − r 1 ) 2 = 化 i = 1 ∑ m j = 1 ∑ r ( α i j ∗ 4 − 2 + r m )
得到最简式
Q = ∑ i = 1 m ∑ j = 1 r α i j ∗ 4 Q=\sum_{i=1}^{m} \sum_{j=1}^{r}{\alpha_{i j}^{*}}^4
Q = i = 1 ∑ m j = 1 ∑ r α i j ∗ 4
这里的 1 / r 1/r 1 / r
基本思想： 同时考虑载荷矩阵每行的方差和每列的方差，对其加权平均最大化。
V V V
V = − 1 m ∑ j = 1 r ( ∑ i = 1 m d i j ) 2 , ∑ i = 1 m d i j 2 ≈ 1 , d i j ≈ α i j ∗ V=-\frac{1}{m}\sum_{j=1}^{r}{\left(\sum_{i=1}^{m} d_{ij}\right)}^{2}\quad\color{Gray} \tiny ,\sum_{i=1}^{m} d_{ij}^2 \approx 1 ,\quad d_{ij}\approx \alpha_{i j}^{*}
V = − m 1 j = 1 ∑ r ( i = 1 ∑ m d i j ) 2 , i = 1 ∑ m d i j 2 ≈ 1 , d i j ≈ α i j ∗
Q Q Q
Q = ∑ i = 1 m ∑ j = 1 r α i j ∗ 4 , 1 r ≈ ∑ j = 1 r d i j 2 , d i j ≈ α i j ∗ Q=\sum_{i=1}^{m} \sum_{j=1}^{r}{\alpha_{i j}^{*}}^4\color{Gray}\quad \tiny ,\frac{1}{r}\approx \sum_{j=1}^rd_{ij}^2 ,\quad d_{ij}\approx \alpha_{i j}^{*}
Q = i = 1 ∑ m j = 1 ∑ r α i j ∗ 4 , r 1 ≈ j = 1 ∑ r d i j 2 , d i j ≈ α i j ∗
构造
E = ∑ i = 1 m ∑ j = 1 r α i j ∗ 4 − γ m ∑ j = 1 r ( ∑ i = 1 m d i j ) 2 E=\sum_{i=1}^{m} \sum_{j=1}^{r}{\alpha_{i j}^{*}}^4 - \frac{\gamma }{m}\sum_{j=1}^{r}{\left(\sum_{i=1}^{m} d_{ij}\right)}^{2}
E = i = 1 ∑ m j = 1 ∑ r α i j ∗ 4 − m γ j = 1 ∑ r ( i = 1 ∑ m d i j ) 2
其中 γ \gamma γ m 2 \frac{m}{2} 2 m
利用已知的观测值 X \boldsymbol{X} X Λ \boldsymbol{\Lambda} Λ
因子分析的数学模型已经给出，为
[ X 1 X 2 ⋮ X m ] = [ μ 1 μ 2 ⋮ μ m ] + [ α 11 α 12 ⋯ α 1 r α 21 α 22 ⋯ α 2 r ⋮ ⋮ ⋮ α m 1 α m 2 ⋯ α m r ] [ F 1 F 2 ⋮ F r ] + [ ε 1 ε 2 ⋮ ε m ] \left[\begin{array}{c}
X_{1} \\
X_{2} \\
\vdots \\
X_{m}
\end{array}\right]=\left[\begin{array}{c}
\mu_{1} \\
\mu_{2} \\
\vdots \\
\mu_{m}
\end{array}\right]+\left[\begin{array}{cccc}
\alpha_{11} & \alpha_{12} & \cdots & \alpha_{1 r} \\
\alpha_{21} & \alpha_{22} & \cdots & \alpha_{2 r} \\
\vdots & \vdots & & \vdots \\
\alpha_{m 1} & \alpha_{m 2} & \cdots & \alpha_{m r}
\end{array}\right]\left[\begin{array}{c}
F_{1} \\
F_{2} \\
\vdots \\
F_{r}
\end{array}\right]+\left[\begin{array}{c}
\varepsilon_{1} \\
\varepsilon_{2} \\
\vdots\\
\varepsilon_{m}
\end{array}\right] ⎣ ⎢ ⎢ ⎢ ⎡ X 1 X 2 ⋮ X m ⎦ ⎥ ⎥ ⎥ ⎤ = ⎣ ⎢ ⎢ ⎢ ⎡ μ 1 μ 2 ⋮ μ m ⎦ ⎥ ⎥ ⎥ ⎤ + ⎣ ⎢ ⎢ ⎢ ⎡ α 1 1 α 2 1 ⋮ α m 1 α 1 2 α 2 2 ⋮ α m 2 ⋯ ⋯ ⋯ α 1 r α 2 r ⋮ α m r ⎦ ⎥ ⎥ ⎥ ⎤ ⎣ ⎢ ⎢ ⎢ ⎡ F 1 F 2 ⋮ F r ⎦ ⎥ ⎥ ⎥ ⎤ + ⎣ ⎢ ⎢ ⎢ ⎡ ε 1 ε 2 ⋮ ε m ⎦ ⎥ ⎥ ⎥ ⎤
注：此处的 X 1 , X 2 … ; F 1 , F 2 … ; ε 1 , ε 2 … X_1,X_2\dots;F_1,F_2\dots;\varepsilon_1,\varepsilon_2\dots X 1 , X 2 … ; F 1 , F 2 … ; ε 1 , ε 2 … n n n X i = [ x i 1 , x i 2 , ⋯ , x i n ] X_i=[x_{i1},x_{i2},\dotsb,x_{in}] X i = [ x i 1 , x i 2 , ⋯ , x i n ]
原变量 X \boldsymbol{X} X
记因子得分函数为
F j = c j + β j 1 X 1 + ⋯ + β j m X m , j = 1 , 2 , … , r F_j=c_j+\beta_{j1}X_1+\dotsb+\beta_{jm}X_m,\quad j=1,2,\dots,r
F j = c j + β j 1 X 1 + ⋯ + β j m X m , j = 1 , 2 , … , r
要求因子的得分，我们想要得分函数的系数，而因为 m > r m>r m > r
把 X i X_i X i μ i = 0 \mu_i=0 μ i = 0
{ X 1 = α 11 F 1 + α 12 F 2 + ⋯ + α 1 r F r + ε 1 X 2 = α 21 F 1 + α 22 F 2 + ⋯ + α 2 r F r + ε 2 ⋯ X m = α m 1 F 1 + α m 2 F 2 + ⋯ + α m r F r + ε p \left\{\begin{array}{l}
X_{1}=\alpha _{11} F_{1}+\alpha_{12} F_{2}+\cdots+\alpha_{1 r} F_{r}+\varepsilon_{1} \\
X_{2}=\alpha_{21} F_{1}+\alpha_{22} F_{2}+\cdots+\alpha_{2 r} F_{r}+\varepsilon_{2} \\
\cdots \\
X_{m}=\alpha_{\tiny m 1} F_{1}+\alpha_{\tiny m 2} F_{2}+\cdots+\alpha_{\tiny m r} F_{r}+\varepsilon_{p}
\end{array}\right. ⎩ ⎪ ⎪ ⎨ ⎪ ⎪ ⎧ X 1 = α 1 1 F 1 + α 1 2 F 2 + ⋯ + α 1 r F r + ε 1 X 2 = α 2 1 F 1 + α 2 2 F 2 + ⋯ + α 2 r F r + ε 2 ⋯ X m = α m 1 F 1 + α m 2 F 2 + ⋯ + α m r F r + ε p
考虑特殊因子方差相异，用加权的最小二乘法，使
∑ i = 1 m 1 σ i 2 [ ( X i ) − ( α i 1 F 1 ^ + ⋯ + a i r F r ^ ) ] 2 \sum_{i=1}^m\frac{1}{\sigma_i^2}\left[(X_i)-(\alpha_{i1}\hat{F_1}+\dotsb+a_{ir}\hat{F_r})\right]^2
i = 1 ∑ m σ i 2 1 [ ( X i ) − ( α i 1 F 1 ^ + ⋯ + a i r F r ^ ) ] 2
最小的 F 1 ^ , F 2 ^ , … , F r ^ \hat{F_1},\hat{F_2},\dots,\hat{F_r} F 1 ^ , F 2 ^ , … , F r ^
矩阵表示：
X = Λ F + ε \boldsymbol{X}=\boldsymbol{\Lambda}\boldsymbol{F}+\boldsymbol{\varepsilon}
X = Λ F + ε
要使
( X − Λ F ^ ) T D − 1 ( X − Λ F ^ ) (\boldsymbol{X}-\bold{\Lambda}\hat{\boldsymbol{F}})^T\boldsymbol{D}^{-1}(\boldsymbol{X}-\bold{\Lambda}\hat{\boldsymbol{F}})
( X − Λ F ^ ) T D − 1 ( X − Λ F ^ )
即取偏导，令
∂ ϕ ( F ) ∂ F = 2 Λ T ( X − Λ F ) = 0 \frac{\partial \phi(\boldsymbol{F})}{\partial \boldsymbol{F}}=2 \boldsymbol{\Lambda}^{T}(\boldsymbol{X}-\boldsymbol{\Lambda} \boldsymbol{F})=0
∂ F ∂ ϕ ( F ) = 2 Λ T ( X − Λ F ) = 0
达到最小，计算得
F ^ = ( Λ T Λ ) − 1 Λ T X \hat{\boldsymbol{F}}=(\boldsymbol{\Lambda}^T\boldsymbol{\Lambda})^{-1}\boldsymbol{\Lambda}^T\boldsymbol{X}
F ^ = ( Λ T Λ ) − 1 Λ T X
F ^ = ( Λ T D − 1 Λ ) − 1 Λ T D − 1 X \hat{\boldsymbol{F}}=(\boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{\Lambda})^{-1}\boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{X}
F ^ = ( Λ T D − 1 Λ ) − 1 Λ T D − 1 X
这种方法得到的因子得分 F i = [ f i 1 , … , f i n ] F_i=[f_{i1},\dots,f_{in}] F i = [ f i 1 , … , f i n ] Z i = [ z i 1 , … , z i n ] Z_i=[z_{i1},\dots,z_{in}] Z i = [ z i 1 , … , z i n ] f i j = z i j / λ j f_{ij}=z_{ij}/\sqrt{\lambda_j} f i j = z i j / λ j
设
[ X 1 X 2 ⋮ X m ] = [ α 11 α 12 ⋯ α 1 r α 21 α 22 ⋯ α 2 r ⋮ ⋮ ⋮ α m 1 α m 2 ⋯ α m r ] [ F 1 F 2 ⋮ F r ] + [ ε 1 ε 2 ⋮ ε m ] \left[\begin{array}{c}
X_{1} \\
X_{2} \\
\vdots \\
X_{m}
\end{array}\right]=\left[\begin{array}{cccc}
\alpha_{11} & \alpha_{12} & \cdots & \alpha_{1 r} \\
\alpha_{21} & \alpha_{22} & \cdots & \alpha_{2 r} \\
\vdots & \vdots & & \vdots \\
\alpha_{m 1} & \alpha_{m 2} & \cdots & \alpha_{m r}
\end{array}\right]\left[\begin{array}{c}
F_{1} \\
F_{2} \\
\vdots \\
F_{r}
\end{array}\right]+\left[\begin{array}{c}
\varepsilon_{1} \\
\varepsilon_{2} \\
\vdots\\
\varepsilon_{m}
\end{array}\right]
⎣ ⎢ ⎢ ⎢ ⎡ X 1 X 2 ⋮ X m ⎦ ⎥ ⎥ ⎥ ⎤ = ⎣ ⎢ ⎢ ⎢ ⎡ α 1 1 α 2 1 ⋮ α m 1 α 1 2 α 2 2 ⋮ α m 2 ⋯ ⋯ ⋯ α 1 r α 2 r ⋮ α m r ⎦ ⎥ ⎥ ⎥ ⎤ ⎣ ⎢ ⎢ ⎢ ⎡ F 1 F 2 ⋮ F r ⎦ ⎥ ⎥ ⎥ ⎤ + ⎣ ⎢ ⎢ ⎢ ⎡ ε 1 ε 2 ⋮ ε m ⎦ ⎥ ⎥ ⎥ ⎤
记第 j j j
F j = β j 1 X 1 + ⋯ + β j m X m , j = 1 , 2 , … , r F_j=\beta_{j1}X_1+\dotsb+\beta_{jm}X_m,\quad j=1,2,\dots,r
F j = β j 1 X 1 + ⋯ + β j m X m , j = 1 , 2 , … , r
因子载荷矩阵的元素表示了样本和因子的相关系数，有
α i j = γ X i F j = E ( X i F j ) = E [ X i ( β j 1 X 1 + ⋯ + β j m X m ) ] = β j 1 γ i 1 + ⋯ + β j m γ i m \alpha_{ij}=\gamma_{\tiny X_i F_j}=E(X_iF_j)=E[X_i(\beta_{j1}X_1+\dotsb+\beta_{jm}X_m)]=\beta_{j1}\gamma_{i1}+\dotsb+\beta_{jm}\gamma_{im}
α i j = γ X i F j = E ( X i F j ) = E [ X i ( β j 1 X 1 + ⋯ + β j m X m ) ] = β j 1 γ i 1 + ⋯ + β j m γ i m
有
[ γ 11 γ 12 ⋯ γ 1 m γ 21 γ 22 ⋯ γ 2 m ⋮ ⋮ ⋱ ⋮ γ m 1 γ m 2 ⋯ γ m m ] [ β j 1 β j 2 ⋮ β j m ] = [ α 1 j α 2 j ⋮ α m j ] \begin{bmatrix}
\gamma_{11} & \gamma_{12} & \dotsb & \gamma_{1m}\\
\gamma_{21}&\gamma_{22} & \dotsb & \gamma_{2m}\\
\vdots & \vdots & \ddots & \vdots \\
\gamma_{m1}&\gamma_{m2} & \dotsb & \gamma_{mm}
\end{bmatrix}
\begin{bmatrix}
\beta_{j1}\\
\beta_{j2}\\
\vdots\\
\beta_{jm}
\end{bmatrix}=
\begin{bmatrix}
\alpha_{1j}\\
\alpha_{2j} \\
\vdots\\
\alpha_{mj}
\end{bmatrix} ⎣ ⎢ ⎢ ⎢ ⎡ γ 1 1 γ 2 1 ⋮ γ m 1 γ 1 2 γ 2 2 ⋮ γ m 2 ⋯ ⋯ ⋱ ⋯ γ 1 m γ 2 m ⋮ γ m m ⎦ ⎥ ⎥ ⎥ ⎤ ⎣ ⎢ ⎢ ⎢ ⎡ β j 1 β j 2 ⋮ β j m ⎦ ⎥ ⎥ ⎥ ⎤ = ⎣ ⎢ ⎢ ⎢ ⎡ α 1 j α 2 j ⋮ α m j ⎦ ⎥ ⎥ ⎥ ⎤
其中三个矩阵分布代表原始变量系数相关系数矩阵 R \boldsymbol{R} R j j j j j j
得
[ β 11 β 12 ⋯ β 1 m β 21 β 22 ⋯ β 2 m ⋮ ⋮ ⋱ ⋮ β m 1 β m 2 ⋯ β r m ] T = R − 1 Λ \begin{bmatrix}
\beta_{11} & \beta_{12} & \dotsb & \beta_{1m}\\
\beta_{21}&\beta_{22} & \dotsb & \beta_{2m}\\
\vdots & \vdots & \ddots & \vdots \\
\beta_{m1}&\beta_{m2} & \dotsb & \beta_{rm}
\end{bmatrix}^T=\boldsymbol{R}^{-1}\boldsymbol{\Lambda}
⎣ ⎢ ⎢ ⎢ ⎡ β 1 1 β 2 1 ⋮ β m 1 β 1 2 β 2 2 ⋮ β m 2 ⋯ ⋯ ⋱ ⋯ β 1 m β 2 m ⋮ β r m ⎦ ⎥ ⎥ ⎥ ⎤ T = R − 1 Λ
因此，因子得分估计为
F ^ = Λ T R − 1 X \hat{\boldsymbol{F}}=\boldsymbol{\Lambda}^T\boldsymbol{R}^{-1}\boldsymbol{X}
F ^ = Λ T R − 1 X
F ^ = Λ T ( Λ Λ T + D ) X \hat{\boldsymbol{F}}=\boldsymbol{\Lambda}^T(\boldsymbol{\Lambda}\boldsymbol{\Lambda}^T+\boldsymbol{D})\boldsymbol{X}
F ^ = Λ T ( Λ Λ T + D ) X
司守奎给出的结果有误
F ^ ( 1 ) = ( Λ T D − 1 Λ ) − 1 Λ T D − 1 X , F ^ ( 2 ) = Λ T ( Λ Λ T + D ) X \begin{array}{l}\hat{\boldsymbol{F}}(1)=(\boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{\Lambda})^{-1}\boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{X},\\
\hat{\boldsymbol{F}}(2)=\boldsymbol{\Lambda}^T(\boldsymbol{\Lambda}\boldsymbol{\Lambda}^T+\boldsymbol{D})\boldsymbol{X}\end{array} F ^ ( 1 ) = ( Λ T D − 1 Λ ) − 1 Λ T D − 1 X , F ^ ( 2 ) = Λ T ( Λ Λ T + D ) X
F ^ ( 1 ) = ( I r + ( Λ T D − 1 Λ ) − 1 ) F ^ ( 2 ) \hat{\boldsymbol{F}}(1)=(I_r+(\boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{\Lambda})^{-1})\hat{\boldsymbol{F}}(2)
F ^ ( 1 ) = ( I r + ( Λ T D − 1 Λ ) − 1 ) F ^ ( 2 )
( Λ T D − 1 Λ ) − 1 (\boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{\Lambda})^{-1} ( Λ T D − 1 Λ ) − 1 F ^ ( 1 ) \hat{\boldsymbol{F}}(1) F ^ ( 1 ) F ^ ( 2 ) \hat{\boldsymbol{F}}(2) F ^ ( 2 ) Λ T D − 1 Λ \boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{\Lambda} Λ T D − 1 Λ 0 0 0
E ( F ^ ( 1 ) ∣ F ) = F E ( F ^ ( 2 ) ∣ F ) = ( I r + ( Λ T D − 1 Λ ) − 1 ) Λ T D − 1 Λ F E(\hat{\boldsymbol{F}}(1)|\boldsymbol{F})=\boldsymbol{F}\\
E(\hat{\boldsymbol{F}}(2)|\boldsymbol{F})=(I_r+(\boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{\Lambda})^{-1})\boldsymbol{\Lambda}^T\boldsymbol{D}^{-1}\boldsymbol{\Lambda}\boldsymbol{F} E ( F ^ ( 1 ) ∣ F ) = F E ( F ^ ( 2 ) ∣ F ) = ( I r + ( Λ T D − 1 Λ ) − 1 ) Λ T D − 1 Λ F
第一种估计无偏，回归估计有偏。
E [ ( F ^ ( 1 ) − F ) ( F ^ ( 1 ) − I F ) ′ ∣ F ] = ( Λ ′ D − 1 Λ ) − 1 E [ ( F ^ ( 2 ) − F ) ( F ^ ( 2 ) − I F ) ′ ∣ F ] = ( I m + Λ ′ D − 1 Λ ) − 1 \mathrm{E}\left[(\hat{\boldsymbol{F}}(1)-\boldsymbol{F})(\hat{\boldsymbol{F}}(1)-{ }^{I} \boldsymbol{F})^{\prime} \mid \boldsymbol{F}\right]=\left(\boldsymbol{\Lambda}^{\prime} \boldsymbol{D}^{-1} \boldsymbol{\Lambda}\right)^{-1} \\
\mathrm{E}\left[(\hat{\boldsymbol{F}}(2)-\boldsymbol{F})(\hat{\boldsymbol{F}}(2)-{ }^{I} \boldsymbol{F})^{\prime} \mid \boldsymbol{F}\right]=\left(I_{m}+\boldsymbol{\Lambda}^{\prime} \boldsymbol{D}^{-1} \boldsymbol{\Lambda}\right)^{-1} E [ ( F ^ ( 1 ) − F ) ( F ^ ( 1 ) − I F ) ′ ∣ F ] = ( Λ ′ D − 1 Λ ) − 1 E [ ( F ^ ( 2 ) − F ) ( F ^ ( 2 ) − I F ) ′ ∣ F ] = ( I m + Λ ′ D − 1 Λ ) − 1
这表示第二种估计（汤普森因子得分）有较小的平均预报误差.
因子分析是十分主观的，在许多出版的资料中，因子分析模型都用少数可命名因子提供了合理解释。实际上，绝大多数因子分析并没有产生如此明确的结果。评价因子分析质量的法则尚未很好量化，因子分析的质量参差不齐。
本文章花费大量时间查询比较各版本的过程，最终以司守奎的参数名为主体，调整了部分参数命名以符合笔者习惯。
本文章的格式不清晰，多次差错后依然有公式格式，变量名错误的问题，还请见谅。
