Main Content

Fully Independent Conditional Approximation for GPR Models

The fully independent conditional (FIC) approximation [1] is a way of systematically approximating the true GPR kernel function in a way that avoids the predictive variance problem of the SR approximation while still maintaining a valid Gaussian process. You can specify the FIC method for parameter estimation by using the 'FitMethod','fic' name-value pair argument in the call to fitrgp. For prediction using FIC, you can use the 'PredictMethod','fic' name-value pair argument in the call to fitrgp.

Approximating the Kernel Function

The FIC approximation to k(xi,xj|θ) for active set AN={1,2,...,n} is given by:

k^FIC(xi,xj|θ,A)=k^SR(xi,xj|θ,A)+δij(k(xi,xj|θ)k^SR(xi,xj|θ,A)),δij={1,ifi=j,0ifij.

That is, the FIC approximation is equal to the SR approximation if ij. For i=j, the software uses the exact kernel value rather than an approximation. Define an n-by-n diagonal matrix Ω(X|θ,A) as follows:

[Ω(X|θ,A)]ij=δij(k(xi,xj|θ)k^SR(xi,xj|θ,A))={k(xi,xj|θ)k^SR(xi,xj|θ,A)ifi=j,0ifij.

The FIC approximation to K(X,X|θ) is then given by:

K^FIC(X,X|θ,A)=K^SR(X,X|θ,A)+ Ω(X|θ,A)= K(X,XA|θ)K(XA,XA|θ)1K(XA,X|θ)+Ω(X|θ,A).

Parameter Estimation

Replacing K(X,X|θ) by K^FIC(X,X|θ,A) in the marginal log likelihood function produces its FIC approximation:

logPFIC(y|X,β,θ,σ2,A)=12(yHβ)T[K^FIC(X,X|θ,A)+σ2In]1(yHβ)N2log2π12log|K^FIC(X,X|θ,A)+σ2In|.

As in the exact method, the software estimates the parameters by first computing β^(θ,σ2), the optimal estimate of β, given θ and σ2. Then it estimates θ, and σ2 using the β-profiled marginal log likelihood. The FIC estimate to β for given θ, and σ2 is

β^FIC(θ,σ2,A)=[HT(K^FIC(X,X|θ,A)+σ2 IN)1H*]1HT(K^FIC(X,X|θ,A)+σ2 IN)1y**,

*=HTΛ(θ,σ2,A)1HHTΛ(θ,σ2,A)1K(X,XA|θ)BA1K(XA,X|θ)Λ(θ,σ2,A)1H,**=HTΛ(θ,σ2,A)1yHTΛ(θ,σ2,A)1K(X,XA|θ)BA1K(XA,X|θ)Λ(θ,σ2,A)1y,BA=K(XA,XA|θ)+K(XA,X|θ)Λ(θ,σ2,A)1K(X,XA|θ),Λ(θ,σ2,A)=Ω(X|θ,A)+σ2In.

Using β^FIC(θ,σ2,A), the β-profiled marginal log likelihood for FIC approximation is:

logPFIC(y|X,β^FIC(θ,σ2,A),θ,σ2,A)=12(yHβ^FIC(θ,σ2,A))T(K^FIC(X,X|θ,A)+σ2IN)1(yHβ^FIC(θ,σ2,A))N2log2π12log|K^FIC(X,X|θ,A)+σ2IN|,

where

(K^FIC(X,X|θ,A)+σ2IN)1=Λ(θ,σ2,A)1Λ(θ,σ2,A)1K(X,XA|θ)BA1K(XA,X|θ)Λ(θ,σ2,A)1,log|K^FIC(X,X|θ,A)+σ2IN|=log|Λ(θ,σ2,A)|+log|BA|log|K(XA,XA|θ)|.

Prediction

The FIC approximation to the distribution of ynew given y, X, xnew is

P(ynew|y,X,xnew)=N(ynew|h(xnew)Tβ+μFIC,σnew2+ΣFIC),

where μFIC and ΣFIC are the FIC approximations to μ and Σ given in prediction using exact GPR method. As in the SR case, μFIC and ΣFIC are obtained by replacing all occurrences of the true kernel with its FIC approximation. The final forms of μFIC and ΣFIC are as follows:

μFIC= K(xnewT,XA|θ) BA1 K(XA,X|θ) Λ(θ,σ2,A)1(yHβ),

ΣFIC=k(xnew,xnew|θ)K(xnewT,XA|θ)K(XA,XA|θ)1K(XA,xnewT|θ)+K(xnewT,XA|θ)BA1K(XA,xnewT|θ),

where

BA=K(XA,XA|θ)+K(XA,X|θ)Λ(θ,σ2,A)1K(X,XA|θ),Λ(θ,σ2,A)=Ω(X|θ,A)+σ2In.

References

[1] Candela, J. Q. "A Unifying View of Sparse Approximate Gaussian Process Regression." Journal of Machine Learning Research. Vol 6, pp. 1939–1959, 2005.

See Also

|

Related Topics