NB. statistical functions

NB.*mean v         arithmetic mean
NB.*geomean v      geometric mean
NB.*harmean v      harmonic mean
NB.
NB.*dev v          deviation from mean
NB.*ssdev v        sum squares of deviation
NB.*var v          variance
NB.*stddev v       standard deviation
NB.*skewness v     skewness
NB.*kurtosis v     kurtosis
NB.
NB.*spdev v        sum of products of deviations
NB.*cov v          covariance
NB.*corr v         correlation
NB.
NB.*min v          minimum
NB.*max v          maximum
NB.*midpt v        index of midpoint
NB.*median v       median
NB.
NB.*cile v         x. cile values of y.
NB.*comb v         combinations of size x. from i.y
NB.*combrev v      comb in revolving door order
NB.*dstat v        descriptive statistics
NB.*freqcount v    frequency count
NB.*lsfit v        least-squares fit
NB.*perm v         permutations of size y.
NB.*regression v   multiple regression

mean=: +/ % #
geomean=: */ %:~ #
harmean=: mean &. (%"_)

dev=: -"_1 _ mean
ssdev=: +/ @: *: @ dev
var=: ssdev % <:@#
stddev=: %: @ var

NB. =========================================================
NB. skewness = 3rd moment coefficient
skewness=: %:@# * +/@(^&3)@dev % ^&1.5@ssdev

NB. =========================================================
NB. kurtosis = 4th moment coefficient
kurtosis=: # * +/@(^&4)@dev % *:@ssdev

spdev=: +/ @ (*~ dev)
cov=: spdev % <: @ # @ ]
corr=: cov % * & stddev

min=: <./
max=: >./
midpt=: -:@<:@#
median=: -:@(+/)@((<. , >.)@midpt { /:~)

NB. =========================================================
NB. cile    e.g. 3 cile i.12
cile=: $@] $ ((* <.@:% #@]) /:@/:@,)

NB. =========================================================
NB. comb
NB. all combinations of size x. from i.y.
comb=: 4 : 0
k=. i.>:d=. y.-x.
z=. (d$<i.0 0),<i.1 0
for. i.x. do. z=. k ,.&.> ,&.>/\. >:&.> z end.
; z
)

NB. =========================================================
NB. combrev
NB.
NB. combinations in what Knuth calls "revolving door order"
NB. such that any two adjacent combinations differ by a
NB. single element (Gray codes for combinations).
NB. thanks to Andrew Nikitin (J Forum 27 Oct 2002)
combrev=: 4 : 0
i=. 1+x.
z=. 1 0$k=. i.#c=. 1,~(y.-x.)$0
while. i=. <:i do. z=. (c#k),.;(-c=. +/\.c)|.@{.&.><1+z end.
|.y.-1+z
)

NB. =========================================================
NB. descriptive statistics
dstat=: 3 : 0
t=. '/sample size/minimum/maximum/median/mean'
t=. t,'/std devn/skewness/kurtosis'
t=. ,&':  ' ;._1 t
v=. $,min,max,median,mean,stddev,skewness,kurtosis
t,. ": ,. v y.
)

NB. =========================================================
NB. frequency count (value, frequency) sorted by decreasing frequency
freqcount=: (\: {:"1)@(~. ,. #/.~)

NB. =========================================================
NB. lsfit
NB. n lsfit xy
NB. coefficients of polynomial fitting data points
NB. using least squares approximation.
NB. xy = 2 row matrix of x ,: y
NB. n  = order of polynomial
lsfit=: {:@] %. {.@] ^/ i.@>:@[

NB. =========================================================
NB. all permutations of size y.
perm=: ! A.&i. ]

NB. =========================================================
NB. regression
NB. syntax:  independent regression dependent
NB.    dependent = vector of n observations (Y value)
NB.  independent = n by p matrix of n observations for p independent
NB.                variables (X value)
NB.
NB. returns formatted values
regression=: 4 : 0
x=. 1,.x.
y=. y.
b=. y %.x
k=. <:{:$x
n=. $y
sst=. +/*:y-(+/y) % #y
sse=. +/*:y-x +/ .* b
mse=. sse%n->:k
seb=. %:({.mse)*(<0 1)|:%.(|:x) +/ .* x
ssr=. sst-sse
msr=. ssr%k
rsq=. ssr%sst
F=. msr%mse

r=. ,: '             Var.       Coeff.         S.E.           t'
r=. r, 15.0 15.5 15.5 12.2 ": (i.>:k),.b,.seb,.b%seb
r=. r, ' '
r=. r, '  Source     D.F.        S.S.          M.S.           F'
r=. r, 'Regression', 5.0 15.5 15.5 12.2 ": k, ssr,msr,F
r=. r, 'Error     ', 5.0 15.5 15.5": (n-k+1), sse,mse
r=. r, 'Total     ', 5.0 15.5 ": (n-1), sst
r=. r, ' '
r=. r, 'S.E. of estimate    ', 12.5":%:mse
r=. r, 'Corr. coeff. squared', 12.5": rsq
)
