最小二乘法是一种优化算法,最小二乘法名字的缘由有两个:一是要将误差最小化,二是将误差最小化的方法是使误差的平方和最小化。利用最小二乘法可以简便地求得未知的数据,并使得这些求得的数据与实际数据之间误差的平方和为最小。最小二乘法还可用于曲线拟合,所拟合的曲线可以是线性拟合与非线性拟合。
--------------------一元线性函数--------------------
形式1:借用案例(http://blog.csdn.net/qll125596718/article/details/8248249)首先以一元线性方程参数估计为例,样本回归模型:
残差平方和:
则通过Q最小确定这条直线,即确定,以为变量,把它们看作是Q的函数,就变成了一个求极值的问题,可以通过求导数得到。求Q对两个待估参数的偏导数:
解得:
实例(c++):
double x[] = { 1, 2, 3, 4, 5, 6 }; double y[] = { 3, 5.5, 6.8, 8.8, 11, 12}; valarray<double> data_x(x, 6); valarray<double> data_y(y, 6); A = (data_x*data_x).sum(); C = (data_x*data_y).sum(); float tmp = A*data_x.size() - B*B; k = (C*data_x.size() - B*D) / tmp; cout << "y=" << k << "x+" << b << endl;
运行结果:
注:valarray类似vector,也是一个模板类,主要用来对一系列元素进行高速的数字计算,其与vector的主要区别在于以下两点:
1、valarray定义了一组在两个相同长度和相同类型的valarray类对象之间的数字计算;
2、通过重载operater[],可以返回valarray的相关信息(valarray其中某个元素的引用、特定下标的值或者其某个子集)。
valarray类构造函数:
valarray( );
explicit valarray(size_t _Count);
valarray( const Type& _Val, size_t _Count);
valarray( const Type* _Ptr, size_t _Count);
valarray( const slice_array<Type>& _SliceArray);
valarray( const gslice_array<Type>& _GsliceArray);
valarray( const mask_array<Type>& _MaskArray);
valarray( const indirect_array<Type>& _IndArray);
valarray 类用法:
1. apply 将 valarray 数组的每一个值都用 apply 所接受到的函数进行计算
2. cshift 将 valarray 数组的数据进行循环移动,参数为正者左移为负就右移
3. max 返回 valarray 数组的最大值
4. min 返回 valarray 数组的最小值
5. resize 重新设置 valarray 数组大小,并对其进行初始化
6. shift 将 valarray 数组移动,参数为正者左移,为负者右移,移动后由 0 填充剩余位
7. size 得到数组的大小
8. sum 数组求和
--------------------N元线性函数--------------------
一元线性方程可以看做多元函数的特例,现在用矩阵形式表述多元函数情况下,最小二乘的一般形式:
设拟合多项式为:
各点到这条曲线的距离之和,即偏差平方和如下:
对等式右边求ai偏导数,得到:
.......
把这些等式表示成矩阵的形式,就可以得到下面的矩阵:
(3)
进行化简计算:
,
上面公式(3)可以写为:
,
typedef vector<double> doubleVector; vector<point> getFile(char *File); //获取文件数据 doubleVector getCoeff(vector<point> sample, int n); //矩阵方程 doubleVector coefficient; coefficient = getCoeff(sample, n); for (i = 0; i < coefficient.size(); i++) printf("a%d = %lf\n", i, coefficient[i]); doubleVector getCoeff(vector<point> sample, int n) vector<doubleVector> matFunX; //公式3左矩阵 vector<doubleVector> matFunY; //公式3右矩阵 for (k = 0; k < sample.size(); k++) sum += pow(sample[k].x, j + i); //printf("matFunX.size=%d\n", matFunX.size()); //printf("matFunX[3][3]=%f\n", matFunX[3][3]); for (k = 0; k < sample.size(); k++) sum += sample[k].y*pow(sample[k].x, i); printf("matFunY.size=%d\n", matFunY.size()); double num1, num2, ratio; for (i = 0; i < matFunX.size() - 1; i++) for (j = i + 1; j < matFunX.size(); j++) for (k = 0; k < matFunX.size(); k++) matFunX[j][k] = matFunX[j][k] - matFunX[i][k] * ratio; matFunY[j][0] = matFunY[j][0] - matFunY[i][0] * ratio; doubleVector coeff(matFunX.size(), 0); for (i = matFunX.size() - 1; i >= 0; i--) if (i == matFunX.size() - 1) coeff[i] = matFunY[i][0] / matFunX[i][i]; for (j = i + 1; j < matFunX.size(); j++) matFunY[i][0] = matFunY[i][0] - coeff[j] * matFunX[i][j]; coeff[i] = matFunY[i][0] / matFunX[i][i]; vector<point> getFile(char *File) FILE *fp=fopen(File, "r"); printf("Open file error!!!\n"); while (fscanf(fp, "%lf", &num) != EOF)
XY.txt内容:
另外在http://blog.csdn.net/lsh_2013/article/details/46697625里也有相关程序:
double sum(vector<double> Vnum, int n); double MutilSum(vector<double> Vx, vector<double> Vy, int n); double RelatePow(vector<double> Vx, int n, int ex); double RelateMutiXY(vector<double> Vx, vector<double> Vy, int n, int ex); void EMatrix(vector<double> Vx, vector<double> Vy, int n, int ex, double coefficient[]); void CalEquation(int exp, double coefficient[]); double F(double c[],int l,int m); int main(int argc, char* argv[]) double arry1[5]={0,0.25,0,5,0.75}; double arry2[5]={1,1.283,1.649,2.212,2.178}; memset(coefficient,0,sizeof(double)*5); EMatrix(vx,vy,5,3,coefficient); printf("拟合方程为:y = %lf + %lfx + %lfx^2 \n",coefficient[1],coefficient[2],coefficient[3]); double sum(vector<double> Vnum, int n) double MutilSum(vector<double> Vx, vector<double> Vy, int n) double RelatePow(vector<double> Vx, int n, int ex) double RelateMutiXY(vector<double> Vx, vector<double> Vy, int n, int ex) dReMultiSum+=pow(Vx[i],ex)*Vy[i]; void EMatrix(vector<double> Vx, vector<double> Vy, int n, int ex, double coefficient[]) for (int i=1; i<=ex; i++) for (int j=1; j<=ex; j++) Em[i][j]=RelatePow(Vx,n,i+j-2); Em[i][ex+1]=RelateMutiXY(Vx,Vy,n,i-1); CalEquation(ex,coefficient); void CalEquation(int exp, double coefficient[]) for(int k=1;k<exp;k++) //消元过程 for(int i=k+1;i<exp+1;i++) Em[i][j]=Em[i][j]-Em[k][j]*p1; coefficient[exp]=Em[exp][exp+1]/Em[exp][exp]; for(int l=exp-1;l>=1;l--) //回代求解 coefficient[l]=(Em[l][exp+1]-F(coefficient,l+1,exp))/Em[l][l]; double F(double c[],int l,int m)
memset相关介绍:
http://baike.baidu.com/link?url=p7JreiRCj9yPs3r3WAfsXgynjvtGrWoQ_exF9tFGK6fsVP7V6tdm-_13QhCZxqPrfRi0wH0EihhRL_-qVvrewq
http://c./cpp/html/157.html
--------------------拟合圆的方程--------------------
最小二乘法拟合圆,拟合出的圆以圆心坐标和半径的形式表示 typedef complex<int> POINT; bool FitCircle(const std::vector<POINT> &points, double &er_x, double &er_y, double &radius) double sum_x = 0.0f, sum_y = 0.0f; double sum_x2 = 0.0f, sum_y2 = 0.0f; double sum_x3 = 0.0f, sum_y3 = 0.0f; double sum_xy = 0.0f, sum_x1y2 = 0.0f, sum_x2y1 = 0.0f; for (int i = 0; i < N; i++) double x = points[i].real(); double y = points[i].imag(); C = N * sum_x2 - sum_x * sum_x; D = N * sum_xy - sum_x * sum_y; E = N * sum_x3 + N * sum_x1y2 - (sum_x2 + sum_y2) * sum_x; G = N * sum_y2 - sum_y * sum_y; H = N * sum_x2y1 + N * sum_y3 - (sum_x2 + sum_y2) * sum_y; a = (H * D - E * G) / (C * G - D * D); b = (H * C - E * D) / (D * D - G * C); c = -(a * sum_x + b * sum_y + sum_x2 + sum_y2) / N; radius = sqrt(a * a + b * b - 4 * c) / 2;
--------------------拟合椭圆方程--------------------
//LSEllipse.h
/************************************************************************* 功能说明: 对平面上的一些列点给出最小二乘的椭圆拟合,利用奇异值分解法 调用形式: cvFitEllipse2f(arrayx,arrayy,box); 参数说明: arrayx: arrayx[n],每个值为x轴一个点 arrayx: arrayy[n],每个值为y轴一个点 box : box[5],椭圆的五个参数,分别为center.x,center.y,2a,2b,xtheta esp: 解精度,通常取1e-6,这个是解方程用的说 ***************************************************************************/ vector<double> getEllipseparGauss(vector<CPoint> vec_point); void cvFitEllipse2f( int *arrayx, int *arrayy,int n,float *box ); int SVD(float *a,int m,int n,float b[],float x[],float esp); int gmiv(float a[],int m,int n,float b[],float x[],float aa[],float eps,float u[],float v[],int ka); int ginv(float a[],int m,int n,float aa[],float eps,float u[],float v[],int ka); int muav(float a[],int m,int n,float u[],float v[],float eps,int ka);
//LSEllipse.cpp
LSEllipse::LSEllipse(void) LSEllipse::~LSEllipse(void) //A为系数矩阵,x为解向量,若成功,返回true,否则返回false,并将x清空。 bool RGauss(const vector<vector<double> > & A, vector<double> & x) vector<vector<double> > Atemp(n); for (int i = 0; i < n; i++) for (int j = 0; j < m; j++) for (int k = 0; k < n; k++) for (int i = k; i < n; i++) if (abs(Atemp[i][k]) > max) for (int i = 0; i < m; i++) double temp = Atemp[l][i]; Atemp[l][i] = Atemp[k][i]; for (int i = k+1; i < n; i++) double l = Atemp[i][k]/Atemp[k][k]; for (int j = k; j < m; j++) Atemp[i][j] = Atemp[i][j] - l*Atemp[k][j]; x[n-1] = Atemp[n-1][m-1]/Atemp[n-1][m-2]; for (int k = n-2; k >= 0; k--) for (int j = k+1; j < n; j++) x[k] = (Atemp[k][m-1] - s)/Atemp[k][k]; vector<double> LSEllipse::getEllipseparGauss(vector<CPoint> vec_point) vector<double> vec_result; double x3y1 = 0,x1y3= 0,x2y2= 0,yyy4= 0, xxx3= 0,xxx2= 0,x2y1= 0,yyy3= 0,x1y2= 0 ,yyy2= 0,x1y1= 0,xxx1= 0,yyy1= 0; int N = vec_point.size(); for (int m_i = 0;m_i < N ;++m_i ) double xi = vec_point[m_i].x ; double yi = vec_point[m_i].y; long double Bb[5],Cc[5],Dd[5],Ee[5],Aa[5]; Bb[0] = x1y3, Cc[0] = x2y1, Dd[0] = x1y2, Ee[0] = x1y1, Aa[0] = x2y2; Bb[1] = yyy4, Cc[1] = x1y2, Dd[1] = yyy3, Ee[1] = yyy2, Aa[1] = x1y3; Bb[2] = x1y2, Cc[2] = xxx2, Dd[2] = x1y1, Ee[2] = xxx1, Aa[2] = x2y1; Bb[3] = yyy3, Cc[3]= x1y1, Dd[3] = yyy2, Ee[3] = yyy1, Aa[3] = x1y2; Bb[4]= yyy2, Cc[4]= xxx1, Dd[4] = yyy1, Ee[4] = N, Aa[4]= x1y1; vector<vector<double>>Ma(5); Ma[i].push_back(resul[i]); double XC=(2*B*C-A*D)/(A*A-4*B); double YC=(2*D-A*C)/(A*A-4*B); long double fenzi=2*(A*C*D-B*C*C-D*D+4*E*B-A*A*E); long double fenmu=(A*A-4*B)*(B-sqrt(A*A+(1-B)*(1-B))+1); long double fenmu2=(A*A-4*B)*(B+sqrt(A*A+(1-B)*(1-B))+1); double XA=sqrt(fabs(fenzi/fenmu)); double XB=sqrt(fabs(fenzi/fenmu2)); double Xtheta=0.5*atan(A/(1-B))*180/3.1415926; vec_result.push_back(XC); vec_result.push_back(YC); vec_result.push_back(XA); vec_result.push_back(XB); vec_result.push_back(Xtheta); void LSEllipse::cvFitEllipse2f( int *arrayx, int *arrayy,int n,float *box ) float *A1=new float[n*5]; float *A2=new float[2*2]; float *A3=new float[n*3]; float *B1=new float[n],*B2=new float[2],*B3=new float[n]; const double min_eps = 1e-6; //解出Ax^2+By^2+Cxy+Dx+Ey=10000的最小二乘解! SVD(A1,n,5,B1,x1,min_eps); A2[0]=2*x1[0],A2[1]=A2[2]=x1[2],A2[3]=2*x1[1]; //标准化,将一次项消掉,求出center.x和center.y; SVD(A2,2,2,B2,x2,min_eps); A3[step] = (px - rp[0]) * (px - rp[0]); A3[step+1] = (py - rp[1]) * (py - rp[1]); A3[step+2] = (px - rp[0]) * (py - rp[1]); //求出A(x-center.x)^2+B(y-center.y)^2+C(x-center.x)(y-center.y)的最小二乘解 SVD(A3,n,3,B3,x1,min_eps); rp[4] = -0.5 * atan2(x1[2], x1[1] - x1[0]); if( fabs(t) > fabs(x1[2])*min_eps ) rp[2] = fabs(x1[0] + x1[1] - t); rp[2] = sqrt(2.0 / rp[2]); rp[3] = fabs(x1[0] + x1[1] + t); rp[3] = sqrt(2.0 / rp[3]); box[0] = (float)rp[0] + cx; box[1]= (float)rp[1] + cy; box[2]= (float)(rp[2]*2); box[3] = (float)(rp[3]*2); box[4] = (float)(90 + rp[4]*180/3.1415926); int LSEllipse::SVD(float *a,int m,int n,float b[],float x[],float esp) flag=gmiv(a,m,n,b,x,aa,esp,u,v,ka); int LSEllipse::gmiv( float a[],int m,int n,float b[],float x[],float aa[],float eps,float u[],float v[],int ka) i=ginv(a,m,n,aa,eps,u,v,ka); x[i]=x[i]+aa[i*m+j]*b[j]; int LSEllipse::ginv(float a[],int m,int n,float aa[],float eps,float u[],float v[],int ka) // int muav(float a[],int m,int n,float u[],float v[],float eps,int ka); i=muav(a,m,n,u,v,eps,ka); while ((k<=j)&&(a[k*n+k]!=0.0)) k=k+1; { f=l*n+i; p=j*m+l; q=l*n+l; aa[t]=aa[t]+v[f]*u[p]/a[q]; int LSEllipse::muav(float a[],int m,int n,float u[],float v[],float eps,int ka) { int i,j,k,l,it,ll,kk,ix,iy,mm,nn,iz,m1,ks; float d,dd,t,sm,sm1,em1,sk,ek,b,c,shh,fg[2],cs[2]; void ppp(float a[],float e[],float s[],float v[],int m,int n); void sss(float fg[],float cs[]); s=(float *) malloc(ka*sizeof(float)); e=(float *) malloc(ka*sizeof(float)); w=(float *) malloc(ka*sizeof(float)); { for (kk=1; kk<=ll; kk++) { ix=(i-1)*n+kk-1; d=d+a[ix]*a[ix];} { s[kk-1]=(float)fabs(s[kk-1]); if (a[ix]<0.0) s[kk-1]=-s[kk-1]; { for (j=kk+1; j<=n; j++) { if ((kk<=k)&&(s[kk-1]!=0.0)) { ix=(i-1)*m+kk-1; iy=(i-1)*n+kk-1; { e[kk-1]=(float)fabs(e[kk-1]); if (e[kk]<0.0) e[kk-1]=-e[kk-1]; if ((kk+1<=m)&&(e[kk-1]!=0.0)) { for (i=kk+1; i<=m; i++) w[i-1]=0.0; w[i-1]=w[i-1]+e[j-1]*a[(i-1)*n+j-1]; a[ix]=a[ix]-w[i-1]*e[j-1]/e[kk]; if (l+1<mm) e[l]=a[l*n+mm-1]; { for (j=k+1; j<=nn; j++) { for (ll=1; ll<=k; ll++) { kk=k-ll+1; iz=(kk-1)*m+kk-1; { ix=(i-1)*m+kk-1; u[ix]=-u[ix];} { kk=n-ll+1; iz=kk*n+kk-1; if ((kk<=l)&&(e[kk-1]!=0.0)) { for (j=kk+1; j<=n; j++) { ix=(i-1)*n+kk-1; iy=(i-1)*n+j-1; { ix=(i-1)*n+j-1; iy=(i-1)*n+kk-1; free(s); free(e); free(w); return(1); free(s); free(e); free(w); return(-1); while ((kk!=0)&&(fabs(e[kk-1])!=0.0)) { d=(float)(fabs(s[kk-1])+fabs(s[kk])); { ix=(i-1)*n+kk-1; v[ix]=-v[ix];} while ((kk!=m1)&&(s[kk-1]<s[kk])) { d=s[kk-1]; s[kk-1]=s[kk]; s[kk]=d; { ix=(i-1)*n+kk-1; iy=(i-1)*n+kk; d=v[ix]; v[ix]=v[iy]; v[iy]=d; { ix=(i-1)*m+kk-1; iy=(i-1)*m+kk; d=u[ix]; u[ix]=u[iy]; u[iy]=d; while ((ks>kk)&&(fabs(s[ks-1])!=0.0)) if (ks!=mm) d=d+(float)fabs(e[ks-1]); if (ks!=kk+1) d=d+(float)fabs(e[ks-2]); sm=s[mm-1]/d; sm1=s[mm-2]/d; sk=s[kk-1]/d; ek=e[kk-1]/d; b=((sm1+sm)*(sm1-sm)+em1*em1)/2.0f; c=sm*em1; c=c*c; shh=0.0; { shh=(float)sqrt(b*b+c); fg[0]=(sk+sm)*(sk-sm)-shh; fg[0]=cs[0]*s[i-1]+cs[1]*e[i-1]; e[i-1]=cs[0]*e[i-1]-cs[1]*s[i-1]; if ((cs[0]!=1.0)||(cs[1]!=0.0)) d=cs[0]*v[ix]+cs[1]*v[iy]; v[iy]=-cs[1]*v[ix]+cs[0]*v[iy]; fg[0]=cs[0]*e[i-1]+cs[1]*s[i]; s[i]=-cs[1]*e[i-1]+cs[0]*s[i]; if ((cs[0]!=1.0)||(cs[1]!=0.0)) d=cs[0]*u[ix]+cs[1]*u[iy]; u[iy]=-cs[1]*u[ix]+cs[0]*u[iy]; fg[1]=e[mm-2]; e[mm-2]=0.0; for (ll=kk; ll<=mm-1; ll++) if ((cs[0]!=1.0)||(cs[1]!=0.0)) d=cs[0]*v[ix]+cs[1]*v[iy]; v[iy]=-cs[1]*v[ix]+cs[0]*v[iy]; if ((cs[0]!=1.0)||(cs[1]!=0.0)) d=cs[0]*u[ix]+cs[1]*u[iy]; u[iy]=-cs[1]*u[ix]+cs[0]*u[iy]; void ppp(float a[],float e[],float s[],float v[],int m,int n) if (m<n) a[(i-1)*n+i]=e[i-1]; { p=(i-1)*n+j-1; q=(j-1)*n+i-1; d=v[p]; v[p]=v[q]; v[q]=d; void sss(float fg[],float cs[]) if ((fabs(fg[0])+fabs(fg[1]))==0.0) { cs[0]=1.0; cs[1]=0.0; d=0.0;} { d=(float)sqrt(fg[0]*fg[0]+fg[1]*fg[1]); if (fabs(fg[0])>fabs(fg[1])) if (fabs(fg[1])>=fabs(fg[0])) cs[0]=fg[0]/d; cs[1]=fg[1]/d; if (fabs(fg[0])>fabs(fg[1])) r=cs[1]; if (cs[0]!=0.0) r=1.0f/cs[0];
参考:
线性,非线性多项式:
http://blog.csdn.net/qll125596718/article/details/8248249
http://blog.csdn.net/poxiaozhuimeng/article/details/41117947
http://blog.csdn.net/ouyangying123/article/details/53996403
http://blog.csdn.net/jairuschan/article/details/7517773/
http://blog.csdn.net/zang141588761/article/details/50523036
http://www.cnblogs.com/gnuhpc/archive/2012/12/09/2809997.html
http://download.csdn.net/download/biaobiao11/9755119
圆拟合:
http://blog.sina.com.cn/s/blog_b27f71160101gxun.html
http://www.cnblogs.com/dotLive/archive/2007/04/06/524633.html
http://blog.csdn.net/andylao62/article/details/24522365
http://blog.csdn.net/liyuanbhu/article/details/50889951
http://blog.csdn.net/liyuanbhu/article/details/50890587
椭圆拟合:
http://doc./u013708970/archive/121532.html
|