Package 'sfa'

Title: Stochastic Frontier Analysis
Description: Maximum likelihood estimation of stochastic frontier models, in which observed output falls short of an unobserved production frontier by a one-sided inefficiency term in addition to symmetric noise, following Aigner, Lovell and Schmidt (1977) <doi:10.1016/0304-4076(77)90052-5> and Meeusen and van den Broeck (1977) <doi:10.2307/2525757>. Cross-sectional models cover seventeen noise and inefficiency distribution pairs, with heteroskedasticity permitted in either error component. Panel models include the true fixed and random effects estimators of Greene (2005) <doi:10.1016/j.jeconom.2004.05.003> and the four-component model of Colombi, Kumbhakar, Martini and Vittadini (2014) <doi:10.1007/s11123-014-0386-y>. Further entry points address zero inefficiency, latent classes, two-tier frontiers, sample selection, nonparametric frontiers, dependence between the error components, and endogenous regressors after Amsler, Prokhorov and Schmidt (2016) <doi:10.1016/j.jeconom.2015.06.013>. Technical efficiency prediction, marginal effects, and specification tests for wrong skewness, for the assumed inefficiency distribution and for heteroskedasticity are provided.
Authors: David Bernstein [aut, cre] (ORCID: <https://orcid.org/0000-0002-2267-5741>), Christopher Parmeter [aut], Alexander Stead [aut]
Maintainer: David Bernstein <[email protected]>
License: GPL (>= 2)
Version: 1.2.1
Built: 2026-10-14 22:55:10 UTC
Source: https://github.com/davidhbernstein/sfa

Help Index


Stochastic Frontier Analysis

Description

Maximum likelihood estimation of stochastic frontier models, in which observed output falls short of an unobserved production frontier by a one-sided inefficiency term in addition to symmetric noise, following Aigner, Lovell and Schmidt (1977) <doi:10.1016/0304-4076(77)90052-5> and Meeusen and van den Broeck (1977) <doi:10.2307/2525757>. Cross-sectional models cover seventeen noise and inefficiency distribution pairs, with heteroskedasticity permitted in either error component. Panel models include the true fixed and random effects estimators of Greene (2005) <doi:10.1016/j.jeconom.2004.05.003> and the four-component model of Colombi, Kumbhakar, Martini and Vittadini (2014) <doi:10.1007/s11123-014-0386-y>. Further entry points address zero inefficiency, latent classes, two-tier frontiers, sample selection, nonparametric frontiers, dependence between the error components, and endogenous regressors after Amsler, Prokhorov and Schmidt (2016) <doi:10.1016/j.jeconom.2015.06.013>. Technical efficiency prediction, marginal effects, and specification tests for wrong skewness, for the assumed inefficiency distribution and for heteroskedasticity are provided.

Details

The DESCRIPTION file:

Package: sfa
Version: 1.2.1
Date: 2026-09-07
Title: Stochastic Frontier Analysis
Type: Package
Authors@R: c(person("David", "Bernstein", email = "[email protected]", role = c("aut", "cre"), comment = c(ORCID = "0000-0002-2267-5741")), person("Christopher", "Parmeter", role = c("aut")), person("Alexander", "Stead", role = c("aut")))
Maintainer: David Bernstein <[email protected]>
Description: Maximum likelihood estimation of stochastic frontier models, in which observed output falls short of an unobserved production frontier by a one-sided inefficiency term in addition to symmetric noise, following Aigner, Lovell and Schmidt (1977) <doi:10.1016/0304-4076(77)90052-5> and Meeusen and van den Broeck (1977) <doi:10.2307/2525757>. Cross-sectional models cover seventeen noise and inefficiency distribution pairs, with heteroskedasticity permitted in either error component. Panel models include the true fixed and random effects estimators of Greene (2005) <doi:10.1016/j.jeconom.2004.05.003> and the four-component model of Colombi, Kumbhakar, Martini and Vittadini (2014) <doi:10.1007/s11123-014-0386-y>. Further entry points address zero inefficiency, latent classes, two-tier frontiers, sample selection, nonparametric frontiers, dependence between the error components, and endogenous regressors after Amsler, Prokhorov and Schmidt (2016) <doi:10.1016/j.jeconom.2015.06.013>. Technical efficiency prediction, marginal effects, and specification tests for wrong skewness, for the assumed inefficiency distribution and for heteroskedasticity are provided.
Suggests: knitr, rmarkdown, testthat, sn, np, DEA, sandwich, lmtest, texreg, pbapply, gsl
Imports: pso, numDeriv, plm, minqa, randtoolbox, mnormt, truncnorm, mvtnorm, Formula, methods, parallel, graphics, grDevices
Depends: R (>= 4.0.0)
License: GPL (>= 2)
Language: en-US
URL: https://www.davidharrybernstein.com/software, https://github.com/davidhbernstein/sfa
BugReports: https://github.com/davidhbernstein/sfa/issues
LazyLoad: yes
NeedsCompilation: no
VignetteBuilder: knitr
Config/testthat/edition: 3
Config/testthat/parallel: true
Config/pak/sysreqs: make
Repository: https://davidhbernstein.r-universe.dev
Date/Publication: 2026-10-14 22:03:28 UTC
RemoteUrl: https://github.com/davidhbernstein/sfa
RemoteRef: HEAD
RemoteSha: 6aaaa4cd388d0c8f78b572696571d6ce28b0e9bc
Author: David Bernstein [aut, cre] (ORCID: <https://orcid.org/0000-0002-2267-5741>), Christopher Parmeter [aut], Alexander Stead [aut]

Index of help topics:

calibrate_c             Fixed robustness parameter by weight matching
copsfm                  Stochastic Frontier with Dependent, and
                        Optionally Skewed, Error Components
cw_test                 Chen and Wang (2012) centered-residuals moment
                        test
data_gen_cs             Generate Cross-Sectional Data for Stochastic
                        Frontier Analysis
data_gen_p              Generate Panel Data for Stochastic Frontier
                        Analysis
density_weights         Density-power weights from a robust frontier
                        fit
efficiency              Technical Efficiency Scores
efficiency_ci           Confidence Intervals for Individual
                        Inefficiency and Efficiency
endogeneity_test        Were the regressors endogenous after all?
esfm                    Stochastic frontier tolerating the wrong
                        skewness
FinnishElec             Expenditure and output of Finnish electricity
                        distribution firms
gof_test                Goodness-of-fit test for the assumed
                        inefficiency distribution
hscore                  Hyvarinen score for a Normal-Half-Normal
                        frontier fit
hscore_select           Select the robustness tuning parameter by
                        Hyvarinen score
Indian                  Paddy farming in Aurepalle, India, 1975-76 to
                        1984-85
inefficiency_test       Test whether there is any technical
                        inefficiency
influence_sfa           Which observations are driving this frontier
                        fit?
ivsfm                   Stochastic Frontier Models with Endogenous
                        Regressors
lcsfm                   Latent Class Stochastic Frontier Models
lcsfm_homogeneity       Test a latent class frontier against
                        homogeneity
marginal_effects        Marginal Effects of the Variance Determinants
                        on Inefficiency
meanefficiency          Model-Implied Mean Efficiency
moment_range            What skewness and excess kurtosis can a pair of
                        distributions produce?
npsfm                   Nonparametric Stochastic Frontier Models
panel89                 Costs of U.S. commercial banks, 1989
pcomposed               The distribution of the composed error
pcomposed_model         Composed-error distribution function for any
                        cross-sectional model
PL80_MVTN               Pitt and Lee (1981) Model III: the multivariate
                        truncated normal panel likelihood
predict.sfareg          Predictions, fitted values and residuals for
                        stochastic frontier fits
print.sfareg            Print method for "sfareg" objects
psfm                    Panel Stochastic Frontier Models
psfm_bootstrap          Parametric Bootstrap for 'psfm()' Panel
                        Stochastic Frontier Models
selsfm                  Stochastic Frontier Models with Sample
                        Selection or Technology Choice
sfa_diagnostics         Optimizer Diagnostics for Stochastic Frontier
                        Fits
sfa-package             Stochastic Frontier Analysis
sfa-wrongskew           The wrong-skewness suite: what to do when the
                        residuals lean the wrong way
sfareg-methods          Extractor methods for "sfareg" objects
sfm                     Cross-Sectional Stochastic Frontier Models
sfma                    Average over inefficiency distributions instead
                        of choosing one
simulation_se           Simulation Standard Errors for Simulated-ML
                        Fits
skewness_decomp         Where the skewness of the composed error comes
                        from
skewness_test           Formal tests for wrong skewness in the OLS
                        residuals
spec_test               Is this pair of distributions defensible?
summary.sfareg          Summary method for "sfareg" objects
TIC                     Choosing between non-nested stochastic frontier
                        specifications
tt_boundary_report      Is a collapsed two-tier noise scale the
                        maximum, or did the optimizer stop short?
ttsfm                   Two-Tier Stochastic Frontier Model
uhet_test               Test whether technical inefficiency depends on
                        firm characteristics
USUtilities             U.S. fossil-fuel steam electric utilities,
                        1986-1997
zsfm                    Zero-Inefficiency Stochastic Frontier Models

See Also

https://www.davidharrybernstein.com/software

https://github.com/davidhbernstein/sfa

Examples

## Simple application of the generalized true random effects estimator.
library(sfa)

data_trial <- data_gen_p(t=10,N=100,  rand = 100, 
                         sig_u = 1,   sig_v = 0.3, 
                         sig_r = .2,  sig_h = .4, 
                         cons  = 0.5, beta1 = 0.5,
                         beta2 = 0.5)

psfm(formula    = y_gtre ~ x1 + x2,    
     model_name = "GTRE", 
     data       = data_trial,
     individual = "name",
     PSopt      = FALSE)

Fixed robustness parameter by weight matching

Description

Chooses the robustness tuning parameter so that a residual a given number of scale units from the reference point retains a common share of its clean-data influence, comparably across the robust criteria. This is the transparent alternative to the data-driven selection of hscore_select.

Usage

calibrate_c(sigma_v, sigma_u, method = c("mlqe", "psi", "mdpd", "all"),
            target = 0.10, k = 3, range = c(1e-4, 0.6))

Arguments

sigma_v, sigma_u

Positive scalars at which to calibrate, typically the estimates from a preliminary fit or the values of a simulation design.

method

Robust criterion, or "all" for a named vector.

target

Share of clean-data influence the reference residual should retain. Default 0.10.

k

Reference residual in scale units. Default 3.

range

Search interval for the tuning parameter.

Details

The calibration matches the norm of the observationwise estimating-equation contribution in the intercept, log⁡σv\log\sigma_v and log⁡σu\log\sigma_u directions. Matching on the intercept direction alone would be vacuous: there the criteria share the density-power factor fc−1f′f^{c-1}f', so it would force the same tuning value for all of them by construction. They differ through the Fisher-consistency correction, which enters the scale directions.

Note that σ=(σv2+σu2)1/2\sigma=(\sigma_v^2+\sigma_u^2)^{1/2} is the scale parameter of the composed density, not the standard deviation of the composed error, which is {σv2+(1−2/π)σu2}1/2\{\sigma_v^2+(1-2/\pi)\sigma_u^2\}^{1/2}. The argument k is measured in the former.

The ratio is not monotone, and there can be two answers. For the Fisher-consistency-corrected criteria the influence ratio falls below the target and is then pulled back up again by the integral correction, so it crosses the target twice. At σv=0.30\sigma_v=0.30, σu=0.60\sigma_u=0.60 the Ψ\Psi/MDPD solutions are c=0.2387c=0.2387 and c=0.5385c=0.5385. The larger is returned: it is the calibration reported by Bernstein, Parmeter and Wright, and it lies on the far side of the dip, so the target down-weighting holds there rather than being passed through on the way. Both are in the "roots" attribute; the non-monotonicity is reported rather than resolved silently. MLqE has a single solution, c=0.2166c=0.2166.

"psi" and "mdpd" always calibrate to the same value: the two objectives differ by the factor (1+c)(1+c), which is constant in the parameters and cancels from the ratio being matched.

Value

A named numeric vector of calibrated tuning parameters, carrying an attribute "roots" that lists every solution found for each criterion.

References

Bernstein, D.H., Parmeter, C.F. and Wright, I.A. (2026). On Robust Estimation of the Stochastic Frontier Model. Working paper.

See Also

hscore_select, density_weights

Examples

## the calibration used in Bernstein, Parmeter and Wright
calibrate_c(sigma_v = 0.30, sigma_u = 0.60, method = "all")

Stochastic Frontier with Dependent, and Optionally Skewed, Error Components

Description

Fits a cross-sectional stochastic frontier in which the noise vv and the inefficiency uu may be dependent, with the dependence carried by a copula, and in which the noise itself may be skewed. Every other model in the package assumes independence and a symmetric vv.

Usage

copsfm(formula, data,
       copula = c("gaussian", "fgm", "frank",
                  "clayton", "clayton90", "clayton180", "clayton270",
                  "gumbel", "gumbel90", "gumbel180", "gumbel270",
                  "joe", "joe90", "joe180", "joe270", "independent"),
       inefdec = TRUE,
       udist = c("hnormal", "exponential"),
       vdist = c("normal", "logistic", "glogistic"),
       n_nodes = 128, maxit.bobyqa = 10000, maxit.psoptim = 1000,
       maxit.optim = 1000, start_val = FALSE, PSopt = FALSE,
       optHessian = TRUE, Method = "L-BFGS-B", verbose = FALSE,
       keep_objective = FALSE, rand.psoptim = NULL)

Arguments

formula

Two-sided formula for the frontier, y ~ x1 + x2. A | segment is an error: variance determinants are not yet supported alongside a copula.

data

A data.frame.

copula

The dependence family. See ‘Choosing a family’ below.

family parameter independence at dependence it can express
"gaussian" ρ∈(−1,1)\rho \in (-1,1) 0 both signs, no tail dependence
"fgm" θ∈[−1,1]\theta \in [-1,1] 0 both signs but weak: Spearman ρ=θ/3\rho = \theta/3
"frank" θ∈R\theta \in \mathbb{R} 0 both signs, full range, no tail dependence
"clayton" θ>0\theta > 0 θ→0\theta \to 0 positive, lower-tail dependence
"gumbel" θ≥1\theta \ge 1 1 positive, upper-tail dependence
"joe" θ≥1\theta \ge 1 1 positive, heavier upper tail than Gumbel

Clayton, Gumbel and Joe carry only positive dependence, which is a real restriction: nothing rules out a negative association between noise and inefficiency, and against negatively dependent data those families can only report the independence boundary. The 90 and 270 suffixes are the rotations that reverse the sign, and 180 is the survival copula, which preserves it. So "clayton270" gives lower-tail dependence with a negative association.

"independent" sets c≡1c \equiv 1 and estimates no dependence parameter. It is not a null option: combined with vdist it fits the skewed-noise frontier on its own, which is a different model from anything sfm offers.

inefdec

TRUE for a production frontier, FALSE for cost.

udist

Marginal distribution of the inefficiency uu. "hnormal" is half-normal with scale σu\sigma_u; "exponential" has mean and standard deviation σu\sigma_u.

vdist

Marginal distribution of the noise vv. "normal" is N(0,σv2)N(0, \sigma_v^2). "logistic" and "glogistic" are the generalized logistic of Bonanno and Domma (2022), centred so that E[v]=0E[v] = 0 for every parameter value:

G(v)=(1+e−z)−αv,z=v/δv+ψ(αv)−ψ(1).G(v) = \left(1 + e^{-z}\right)^{-\alpha_v}, \qquad z = v/\delta_v + \psi(\alpha_v) - \psi(1).

"logistic" fixes αv=1\alpha_v = 1 and "glogistic" estimates it, bounded to [0.05,20][0.05, 20]. αv<1\alpha_v < 1 skews the noise negative and αv>1\alpha_v > 1 positive. The reported scale is named delta_v, not sigma_v, because it is not the standard deviation: Var(v)=δv2[ψ′(αv)+ψ′(1)]\mathrm{Var}(v) = \delta_v^2[\psi'(\alpha_v) + \psi'(1)], so a logistic with δv=1\delta_v = 1 has standard deviation π/3=1.814\pi/\sqrt{3} = 1.814.

n_nodes

Gauss–Legendre nodes for the integral over uu.

keep_objective

Logical. Retain the likelihood closure on the fitted object as $objective, so the score-based tools can difference it: vcov(type = "bhhh") (OPG/BHHH standard errors), influence_sfa, the sandwich methods, TIC and vuong. New in 1.2.1. FALSE by default, because a closure carries its enclosing environment, so a fit saved with one serializes the estimation data and the quadrature nodes alongside the results.

maxit.bobyqa, maxit.psoptim, maxit.optim, start_val, PSopt, optHessian, Method, verbose, rand.psoptim

As in sfm.

Details

In brief:

  • The frontier slopes behave normally. The dependence parameter does not: it needs several thousand observations, and at n=2000n = 2000 its sign can come out wrong, taking σu\sigma_u with it (0.57 against a truth of 1 in the design below).

  • Below a crossover between n=400n = 400 and n=1000n = 1000, plain sfm estimates σu\sigma_u with a lower RMSE than the true copula does.

  • Prefer copula = "frank" or "clayton". The other families often return the independence bound on data generated from themselves, and copsfm warns when you pick one.

  • αv\alpha_v (vdist = "glogistic") is consistent but noisy, and below about n=2000n = 2000 it can return its upper bound.

  • Keep n_nodes at 64 or more. At 32 the estimates are biased.

Each point is measured in the subsections below.

The model

The package's taxonomy already writes the general joint density as

fV,U(v,u)=fV(v) fU(u) c(FV(v),FU(u);ρ),f_{V,U}(v,u) = f_V(v)\,f_U(u)\,c(F_V(v), F_U(u); \rho),

with every existing specification setting c≡1c \equiv 1. This function relaxes that, and lets the two marginals vary. Composing over v=ε+Suv = \varepsilon + Su,

fε(ε)=∫0∞fV(ε+Su) fU(u) c ⁣(FV(ε+Su),FU(u))du,f_\varepsilon(\varepsilon) = \int_0^\infty f_V(\varepsilon + Su)\,f_U(u)\, c\!\left(F_V(\varepsilon + Su), F_U(u)\right) du,

a one-dimensional integral evaluated by Gauss–Legendre quadrature on u=t/(1−t)u = t/(1-t) rather than by simulation.

Quadrature nodes

Measured against the closed-form normal/half-normal density at independence, the largest absolute error in log⁡f\log f is 1.3e-2 at 32 nodes, 2.5e-5 at 64, and 8.2e-14 at 128. Sixty-four looks adequate and is not: 1e-5 per observation is 0.02 in a log-likelihood over 2000 points, which is the scale at which competing modes are compared.

Node counts, measured in paired runs. Forty datasets sent to every node count, so the differences are the quadrature and not the draw. Against the 128-node default, the mean difference in ρ^\hat\rho is +0.0002 at 256 nodes (t=1.37t = 1.37) and +0.0046 at 64 (t=1.47t = 1.47) – neither distinguishable – but -0.0783 at 32 (t=−2.79t = -2.79, p=0.008p = 0.008), with per-dataset differences as large as 0.52. So the quadrature has converged by 64, and 32 must not be used as a fast option: it is biased, not merely noisier. For scale, ρ^\hat\rho's own sampling standard deviation is 0.385 at n=600n = 600, which dwarfs every node-count effect here.

Skewed noise, and what it is for

Bonanno and Domma (2022) argue that the “wrong skewness” anomaly is a specification failure rather than a small-sample accident. The third central moment of the composed error is

E[(ε−Eε)3]=J E[(u−Eu)3]+E[(v−Ev)3]+3cov(u2,v)+3J cov(u,v2)−6[E(u)+J E(v)]cov(u,v),E[(\varepsilon - E\varepsilon)^3] = J\,E[(u-Eu)^3] + E[(v-Ev)^3] + 3\mathrm{cov}(u^2, v) + 3J\,\mathrm{cov}(u, v^2) - 6[E(u)+J\,E(v)]\mathrm{cov}(u,v),

so it depends on the skew of vv and on the dependence between vv and uu, not only on the skew of uu. In a model where vv is symmetric and independent of uu the first term is the only one, and a positive residual skew has nowhere to go but σu=0\sigma_u = 0. vdist = "glogistic" gives it somewhere else to go.

Their four specifications are four (copula, vdist) pairs, all with udist = "exponential":

their label call
(I,S) copula = "independent", vdist = "logistic" independence, symmetric vv
(I,A) copula = "independent", vdist = "glogistic" independence, skewed vv
(D,S) copula = "fgm", vdist = "logistic" dependence, symmetric vv
(D,A) copula = "fgm", vdist = "glogistic" dependence, skewed vv

Verification against the closed form, and two corrected moment formulas

The paper derives the composed density in closed form for this case, as a sum of four Gauss hypergeometric terms – Theorem 1 for a production frontier and Theorem 2 for a cost frontier. This function computes it by quadrature instead, and the two agree to about 10−1410^{-14} in log⁡f\log f across αv\alpha_v, θ\theta and both orientations; the check is in test-copsfm.R. Note that the paper's summary moment formulas are not all consistent with its own appendix: Eq. (9)'s dependence term is half what the appendix's cov(U,V)\mathrm{cov}(U,V) implies, and Eq. (10)'s leading term is −2δu3-2\delta_u^3 where an exponential's third central moment is +2δu3+2\delta_u^3. Simulation follows the appendix. The density theorems – which are what is implemented here – are correct.

The same two slips appear in Eq. (10) and Eq. (11) of the earlier production paper (Bonanno, De Giovanni and Domma 2017), and the corrected expressions have since been derived and pinned. Writing D=ψ(2αv)−ψ(αv)D = \psi(2\alpha_v) - \psi(\alpha_v), the covariance is cov(u,v)=θδuδvD/2\mathrm{cov}(u,v) = \theta \delta_u \delta_v D / 2 – twice the published value – so for a production frontier

Var(ϵ)=δu2+δv2[ψ′(αv)+ψ′(1)]−θδuδvD\mathrm{Var}(\epsilon) = \delta_u^2 + \delta_v^2[\psi'(\alpha_v) + \psi'(1)] - \theta \delta_u \delta_v D

and

E[(ϵ−Eϵ)3]=−2δu3+δv3[ψ′′(αv)−ψ′′(1)]+32θδuδv{δuD+δv[ψ′(αv)−ψ′(2αv)−D2]}.E[(\epsilon - E\epsilon)^3] = -2\delta_u^3 + \delta_v^3[\psi''(\alpha_v) - \psi''(1)] + \tfrac{3}{2}\theta \delta_u \delta_v \{\delta_u D + \delta_v[\psi'(\alpha_v) - \psi'(2\alpha_v) - D^2]\}.

The published dependence bracket carries an extra ψ′(αv)+ψ′(1)\psi'(\alpha_v) + \psi'(1) on top of the factor of two. Both corrections rest on the FGM copula's exact factorization E[g(v)h(u)]=E[g]E[h]+θ E[g(v)(1−2Gv)] E[h(u)(1−2Fu)]E[g(v)h(u)] = E[g]E[h] + \theta\, E[g(v)(1-2G_v)]\, E[h(u)(1-2F_u)], whose two generalized-logistic weights are −δvD-\delta_v D and δv2[ψ′(αv)−ψ′(2αv)−D2]\delta_v^2[\psi'(\alpha_v) - \psi'(2\alpha_v) - D^2]; those are verified to 10−910^{-9} in test-wrongskew-suite.R. skewness_decomp computes the three pieces for any fit.

How well the noise shape is estimated

αv\alpha_v is consistent but noisy, and below about n=2000n = 2000 it can return its upper bound. Measured over 12 replications at each size, data generated at αv=2.5\alpha_v = 2.5, σu=0.9\sigma_u = 0.9, δv=0.4\delta_v = 0.4 and (for FGM) θ=0.7\theta = 0.7:

copula nn median α^v\hat\alpha_v IQR on the bound RMSE
independent 1000 2.54 [1.49, 10.21] 3/12 1.88
independent 4000 3.05 [2.38, 3.31] 0/12 1.20
independent 16000 2.77 [2.21, 2.90] 0/12 0.48
FGM 1000 3.53 [1.92, 4.61] 1/12 4.27
FGM 4000 2.78 [2.63, 3.90] 0/12 1.25
FGM 16000 2.63 [2.38, 2.95] 0/12 0.61

σu\sigma_u and δv\delta_v are unaffected throughout – their medians stay within 2% of the truth at every size, as does θ^\hat\theta by n=16000n = 16000 (0.689 against 0.7). So the cost of a poorly determined αv\alpha_v falls on αv\alpha_v alone.

On an individual sample the profile can be monotone all the way to the bound, and the boundary point can genuinely attain a higher likelihood than the truth. On one such sample at n=2000n = 2000 the profile log-likelihood rose from -2627.5 at αv=1.5\alpha_v = 1.5 to -2622.5 at αv=20\alpha_v = 20 (the truth attaining -2626.3), with θ^\hat\theta falling from 0.73 to 0.38 along the way – αv\alpha_v trades off against the dependence parameter and the intercept, not against the two scales. Read a boundary α^v\hat\alpha_v as “the noise is skewed, direction known, magnitude not”, and compare against vdist = "logistic" rather than reporting the number.

Choosing a family

The families differ in two things that matter: whether they admit negative dependence, and where they concentrate it. Gaussian and Frank spread association evenly and have no tail dependence; Clayton concentrates it in the lower tail and Gumbel and Joe in the upper; FGM can only express weak association at all.

Against that, weigh what the next subsection establishes: the dependence parameter is estimated very imprecisely. On a Gaussian design at n=600n = 600 its sampling standard deviation is 0.385 against a truth of 0.5. Offering fifteen families does not make that better, and it makes one thing worse – with fifteen candidates, some family will attain the highest likelihood on independent data by chance. On a sample generated with independent vv and uu, the fitted log-likelihoods across families span only about 0.8, and the best of them reports θ=1.83\theta = 1.83 for Gumbel. Treat a family comparison as descriptive, not as evidence of a dependence structure, and prefer a family chosen for a reason to one chosen by its likelihood.

How well the dependence parameter is estimated

The dependence parameter needs a large sample. This is the thing to know before using it. The frontier slopes behave normally, but ρ\rho does not. Most of these families do not recover their own dependence parameter, and copsfm warns when you pick one. A correct density does not imply an estimable parameter. Measured on 25 samples per family, generated from that family at n=400n = 400 and refitted with it:

family mean estimate (truth) fits on the independence bound
"frank" 5.43 (5) 0%
"clayton" 2.24 (2) 0%
"gumbel" 1.29 (2) 36%
"joe" 1.52 (2) 40%
"clayton270" 0.23 (2) 56%
"gumbel90" 1.26 (2) 60%

On data generated from a Gumbel copula with Spearman ρ=0.685\rho = 0.685 at n=2000n = 2000, every family – including the true one – returns the independence boundary, and their log-likelihoods differ by less than 0.04. The likelihood is close to flat in the dependence parameter. This is a property of the model rather than of the implementation: each density is checked against the second mixed partial of its own CDF, and each sampler against the family\'s theoretical Spearman correlation. The remaining rotations were not measured, and warn that they were not, rather than implying either outcome.

Prefer "frank" or "clayton", and treat an estimate from any other family as exploratory.

On a Gaussian-copula design with true ρ=0.6\rho = 0.6, σU=1\sigma_U = 1, σV=0.4\sigma_V = 0.4:

nn ρ^\hat\rho σ^U\hat\sigma_U σ^V\hat\sigma_V
2000 -0.247 0.571 0.275
8000 0.528 0.921 0.379
20000 0.571 0.971 0.389

It is consistent – the estimates converge on the truth – but at n=2000n = 2000 the sign can come out wrong. The cause is joint identification rather than a bad optimiser: with β\beta and the scales held at their true values the likelihood peaks exactly at ρ=0.6\rho = 0.6, so ρ\rho is well identified conditionally; it is the trade-off against σU\sigma_U, σV\sigma_V and the intercept that defeats it in moderate samples, and the fitted point genuinely attains a higher likelihood than the truth.

Treat ρ^\hat\rho as informative only in large samples, and compare against an independent fit (sfm) before reading anything into it.

And below a few hundred observations, the independent fit is often the better estimate – even when the dependence is real. Fitting the true copula gives a nearly unbiased but very noisy σ^U\hat\sigma_U; ignoring the copula gives a precise but biased one. RMSE of σ^U\hat\sigma_U (truth 1.0) on Gaussian-copula data, 25 replications:

design nn true copula independence
ρ=0.5\rho = 0.5 400 0.431 0.240
ρ=0.5\rho = 0.5 1000 0.151 0.220
ρ=−0.5\rho = -0.5 400 0.251 0.196
ρ=−0.5\rho = -0.5 1000 0.175 0.203

The crossover sits between n=400n = 400 and n=1000n = 1000. Below it the variance cost of estimating ρ\rho jointly with σU\sigma_U exceeds the bias it removes, so plain sfm wins on RMSE; above it copsfm wins. The independence bias does not shrink with nn (about 22% of σU\sigma_U, since it is misspecification rather than noise), while the copula fit's RMSE falls as it should.

Which densities are offered

Only densities that could be verified are offered. Each is checked in test-copsfm.R: it integrates to 1 over the unit square, and it returns exactly 1 at the independence parameter. Frank, Clayton, Gumbel and Joe, whose densities concentrate in a corner where the integral check is loose, are also checked against the second mixed partial of their own CDF, a test a wrong formula cannot pass. The rotations are those same densities reflected.

Value

An object of class "sfareg", with copula, copula_par (NA when copula = "independent"), udist, vdist and alpha_v alongside the usual components. jlms is E[u∣ε]E[u \mid \varepsilon] and exp_u_hat is E[e−u∣ε]E[e^{-u} \mid \varepsilon], both by the same quadrature the likelihood used; efficiency is exp(-jlms), which is what this function has always returned under that name and is not the Battese–Coelli predictor.

References

Smith, M. D. (2008). Stochastic frontier models with dependent error components. The Econometrics Journal, 11(1), 172–192.

Bonanno, G. and Domma, F. (2022). Analytical derivations of new specifications for stochastic frontiers with applications. Mathematics, 10(20), 3876.

See Also

sfm for the independent case.

Examples

set.seed(4)
## n_nodes = 64 rather than the default 128: the quadrature is converged by 64
## (see Details), and this is the slowest entry point in the package.
n   <- 600
x1  <- rnorm(n); x2 <- rnorm(n)
z1  <- rnorm(n); z2 <- 0.6 * z1 + sqrt(1 - 0.6^2) * rnorm(n)
y   <- 0.5 + 0.8 * x1 - 0.4 * x2 + 0.4 * z1 - qnorm((1 + pnorm(z2)) / 2)
dat <- data.frame(y = y, x1 = x1, x2 = x2)

fit <- copsfm(y ~ x1 + x2, data = dat, n_nodes = 64)
fit$out

Chen and Wang (2012) centered-residuals moment test

Description

Tests the distributional assumption on the composed error using the characteristic function of the centred residuals, without ever evaluating the composed-error density or distribution function.

Usage

cw_test(x, model_name = NULL, tau = 1, type = c("cosine", "sine"),
        data = NULL, n_nodes = 256L)

Arguments

x

An "sfareg" fit, or a numeric vector of regression residuals – ordinary least squares residuals are enough.

model_name

The distributional pair under the null. Required when x is a numeric vector; taken from the fit otherwise. One of "NHN", "NE", "NR", "NU", "NGE".

tau

Frequency at which the characteristic function is compared. Chen and Wang suggest a value around 1, and a single frequency: combining several oversizes the test (see Details).

type

"cosine" (recommended) or "sine".

data

The data the model was fitted to, when x is a fit.

n_nodes

Quadrature nodes for the theoretical moments.

Details

Why this test rather than gof_test. It never touches the composed-error PDF or CDF. It compares the empirical characteristic function of the centred residuals with its theoretical value, and that follows from the characteristic functions of the two components separately, which are in closed form for many pairs whose convolution is not – the normal-gamma model being the standard example of a pair with no closed-form composed density.

Centring is what makes it work. A frontier's intercept is not identified separately from E[u]E[u], so the least-squares intercept estimates α−E[u]\alpha - E[u] rather than α\alpha. Centring the residuals cancels that inconsistent intercept outright, which is why the test is valid off an ordinary regression and does not require maximum likelihood. The variance then has to correct for two nuisances – estimating the scale parameters, and the centring itself. Omitting the second understates the variance and oversizes the test.

Use one frequency, not several. Measured here over 500 replications at a nominal 5%, half-normal data:

test nn τ=1\tau=1 τ=1.5\tau=1.5 τ=2\tau=2
cosine 500 0.070 0.054 0.050
cosine 2000 0.046 0.044 0.050
sine 500 0.092 0.064 0.042
sine 2000 0.068 0.062 0.064

Against exponential inefficiency the cosine test's power at τ=1\tau = 1 is 0.231 at n=500n = 500 and 0.952 at n=2000n = 2000, against 0.108 and 0.706 for the sine test. Both of Chen and Wang's conclusions reproduce: the cosine test is the less sensitive to the choice of τ\tau, and the more powerful. Passing several frequencies at once gives rejection rates of 0.09 to 0.15 at a nominal 5%, because the cosine moments at nearby frequencies are close to collinear and the covariance matrix is then near-singular; the authors report the same for their combined τ\tau.

The moment estimator solves the second- and third-moment equations, which for "NHN" and "NE" is the closed form of the paper's equation (32) and is the same inversion sfm(estimator = "cols") uses. A wrongly signed third moment leaves it with no admissible solution, and that is an error rather than a number: see skewness_test.

Value

An object of class "cw_test": the statistic D, its degrees of freedom and p-value, the moment estimates of σv\sigma_v and σu\sigma_u, and the frequencies used.

References

Chen, Y.-T. and Wang, H.-J. (2012). Centered-residuals-based moment estimator and test for stochastic frontier models. Econometric Reviews 31(6), 625–653.

See Also

gof_test, spec_test, skewness_test

Examples

set.seed(4)
n <- 500
X <- matrix(rnorm(n * 3), n, 3)
y <- X %*% c(1, 1, 1) + rnorm(n, 0, 1) - abs(rnorm(n, 0, 1))
e <- residuals(lm(y ~ X))

cw_test(e, "NHN")            # true pair
cw_test(e, "NE")             # wrong inefficiency distribution

Generate Cross-Sectional Data for Stochastic Frontier Analysis

Description

data_gen_cs generates simulated cross-sectional data based on the stochastic frontier model, allowing for different distributional assumptions for the one-sided technical inefficiency error term (uu) and the two-sided idiosyncratic error term (vv). The model has the general form: Y=β0+β1x1+β2x2+v−uY = \beta_0 + \beta_1 x_1 + \beta_2 x_2 + v - u where u≥0u \geq 0 and represents inefficiency. All variants are produced so that the user can select those that they want.

Usage

data_gen_cs(N, rand, sig_u, sig_v, cons, beta1, beta2, a, mu, sig_w = sig_u,
            shape_g = 2, m_nak = 1, mu_ln = -0.5, k_w = 1.5, lam_tsl = 1.5)

Arguments

N

A single integer specifying the number of observations (cross-sectional units).

rand

A single integer to set the seed for the random number generator, ensuring reproducibility.

sig_u

The standard deviation parameter (σu\sigma_u) for the base distribution of the one-sided error term uu.

sig_v

The standard deviation parameter (σv\sigma_v) for the base distribution of the two-sided error term vv.

cons

The value of the constant term (intercept) in the model.

beta1

The coefficient for the x1x_1 variable.

beta2

The coefficient for the x2x_2 variable.

a

The degrees of freedom parameter for the t half-t distribution (u_t and v_t, respectively). Requires the rt function.

mu

The mean parameter (μ\mu) for the normal truncated normal distribution (u_tn). Requires the rtruncnorm function.

sig_w

The standard deviation/scale parameter (σw\sigma_w) for the second one-sided error component ww, used to generate two-tier model columns (see ttsfm). Defaults to sig_u.

shape_g

Shape of the gamma inefficiency draw behind y_pcs_g, which targets sfm(model_name = "NG"). The scale is set to sig_u/shape_g, so E[u]=σuE[u] = \sigma_u regardless of the shape chosen. sfm()'s NG likelihood reports the shape as mu and the scale as sigu, so the values to recover are shape_g and sig_u/shape_g.

m_nak

Nakagami shape mm behind y_pcs_nak, which targets sfm(model_name = "NNAK"). The spread is Ω=σu2\Omega = \sigma_u^2, so σu\sigma_u is the root-mean-square of uu. The default 1 is the Rayleigh case; 0.5 would collapse the Nakagami onto the half-normal and so duplicate y_pcs.

mu_ln

Log-scale mean of the lognormal inefficiency draw behind y_pcs_ln, which targets sfm(model_name = "NLN"). The log-scale standard deviation is sig_u. The default -0.5 with sig_u = 1 gives E[u]=1E[u] = 1.

k_w

Weibull shape behind y_pcs_wb, which targets sfm(model_name = "NW"). The scale is sig_u.

lam_tsl

Skew parameter λ\lambda behind y_pcs_tsl, which targets sfm(model_name = "TSL"). The scale is sig_u; λ→0\lambda \to 0 recovers the exponential case.

Details

The function simulates two explanatory variables, x1x_1 and x2x_2, as transformations of uniform random variables.

The function generates several different frontier models by combining various distributions for uu and vv:

  • **uu Distributions (Inefficiency):** Half-Normal (HN), Truncated Normal (TN), Half-T (HT), Half-Cauchy (HC), Exponential (E), Half-Uniform (HU).

  • **vv Distributions (Idiosyncratic):** Normal (N), t, Cauchy (C).

**Specific Model Outputs (y_pcs variants):**

  • y_pcs: Normal-Half-Normal (N-HN): v∼N(0,σv2)v \sim N(0, \sigma_v^2), u∼∣N(0,σu2)∣u \sim |N(0, \sigma_u^2)|.

  • y_pcs_z: N-HN with Heteroskedastic σu\sigma_u: σu,i=exp⁡(0.9+0.6Zi)\sigma_{u,i} = \exp(0.9 + 0.6 Z_i), where ZZ is a uniform variable.

  • y_pcs_t: T-Half T (T-HT): v∼T(df=a)⋅σvv \sim T(\text{df}=a) \cdot \sigma_v, u∼∣T(df=a)∣⋅σuu \sim |T(\text{df}=a)| \cdot \sigma_u.

  • y_pcs_st: the column that actually matches sfm's THT. y_pcs_t above draws its two components as two independent rt() variates, which shares the degrees of freedom but not the mixing variable, so the composed error is not skew-t and THT cannot recover it. y_pcs_st uses the single common λ∼Γ(a/2,a/2)\lambda \sim \Gamma(a/2, a/2) of Tancredi (2002), i.e. y=h+(ϵ−z)/λy = h + (\epsilon - z)/\sqrt{\lambda}. y_pcs_t is retained unchanged only for backward compatibility.

  • y_pcs_thn: Student's t–half-normal, matching sfm's tHN: v∼T(df=a)⋅σvv \sim T(\text{df}=a) \cdot \sigma_v and u∼∣N(0,σu2)∣u \sim |N(0,\sigma_u^2)|, drawn independently. Heavy-tailed noise with a conventional one-sided term – distinct from both y_pcs_t and y_pcs_st, in which the inefficiency is heavy-tailed too.

  • y_pcs_tn: Normal-Truncated Normal (N-TN): v∼N(0,σv2)v \sim N(0, \sigma_v^2), u∼TN(μ,σu2)u \sim TN(\mu, \sigma_u^2) on [0,∞)[0, \infty).

  • y_pcs_e: Normal-Exponential (N-E): v∼N(0,σv2)v \sim N(0, \sigma_v^2), u∼Exp(ϕ)u \sim Exp(\phi), where ϕ=1/σu\phi = 1/\sigma_u.

  • y_pcs_c: Cauchy-Half Cauchy (C-HC): v∼Cauchy(0,σv)v \sim Cauchy(0, \sigma_v), u∼∣Cauchy(0,σu)∣u \sim |Cauchy(0, \sigma_u)|.

  • y_pcs_u: Normal-Half Uniform (N-HU): v∼N(0,σv2)v \sim N(0, \sigma_v^2), u∼U(0,σu)u \sim U(0, \sigma_u).

  • y_pcs_w: Normal + Cauchy - Half-Normal: v∼N(0,σv2)+Cauchy(0,σv)v \sim N(0, \sigma_v^2) + Cauchy(0, \sigma_v), u∼∣N(0,σu2)∣u \sim |N(0, \sigma_u^2)|. This introduces a composite vv term.

**Note:** The rtruncnorm function is required for y_pcs_tn and loads with the package. In isolation it could be loaded by using library(truncnorm).

Value

A data frame containing NN observations with the following columns:

name

Individual identifier (simply 11 to NN).

cons

The constant term value.

x1

Simulated explanatory variable x1x_1.

x2

Simulated explanatory variable x2x_2.

u, uz, u_t, u_c, u_e, u_u, u_tn, u_st, u_thn

The simulated one-sided error terms under different distributions.

u_r

Rayleigh one-sided error term, for y_pcs_r. Drawn on the convention sfm's "NR" uses, E[u2]=σu2E[u^2] = \sigma_u^2, so the Rayleigh scale is σu/2\sigma_u/\sqrt{2}, E[u]=σuπ/2E[u] = \sigma_u\sqrt{\pi}/2 and Var(u)=(1−π/4)σu2Var(u) = (1-\pi/4)\sigma_u^2.

v, v_t, v_c, v_st, v_thn

The simulated two-sided error terms under different distributions.

lam_st

The common Gamma(a/2, a/2) scale-mixing variable behind y_pcs_st; both of that column's components are divided by its square root, which is what makes the composed error skew-t.

y_pcs, y_pcs_t, y_pcs_st, y_pcs_thn, y_pcs_e, y_pcs_ez, y_pcs_c, y_pcs_u, y_pcs_z, y_pcs_w, y_pcs_tn

The dependent variable YY under the corresponding SFA model distributions.

y_pcs_r

Normal-Rayleigh dependent variable, for sfm's "NR". Use this column, not y_pcs: the normal-Rayleigh and normal-half-normal composed errors are different families, not reparameterizations of one another, and "NR" cannot recover y_pcs's parameters. True values are (σv,σu)=(\sigma_v, \sigma_u) = (sig_v, sig_u).

z

The auxiliary variable used for heteroskedasticity in y_pcs_z, y_pcs_ez, y_tthn_z, and y_zisf_z.

uz_e

Exponential one-sided error term with heteroskedastic scale, for y_pcs_ez.

w_tt, w_tt_hn, wz_hn

The second one-sided error component (ww) for the two-tier columns below, under exponential, homoskedastic half-normal, and heteroskedastic half-normal distributions respectively.

zp

A second auxiliary variable used for heteroskedasticity of σw\sigma_w in y_tthn_z.

y_ttne

Homoskedastic two-tier dependent variable (v+w−uv + w - u, both one-sided components exponential), for ttsfm's "TTNE"/"TTNLS".

y_tthn

Homoskedastic two-tier dependent variable (both one-sided components half-normal), for ttsfm's "TTHN" with no pipes.

y_tthn_z

Heteroskedastic two-tier dependent variable (σu\sigma_u a function of z, σw\sigma_w a function of zp), for ttsfm's "TTHN" with formula ~x1+x2|z|zp.

eff_ind, eff_ind_z

Indicator (1 = efficient, uu forced to 0) used to build the zero-inefficiency columns below.

prob_z_true

The true heteroskedastic "efficient regime" probability used to draw eff_ind_z.

y_zisf

Zero-inefficiency dependent variable with constant efficient-regime probability, for zsfm's "ZISF".

y_zisf_z

Zero-inefficiency dependent variable with z-dependent efficient-regime probability, for zsfm's "ZISF_Z".

u_g, y_pcs_g

Gamma one-sided error term and the corresponding dependent variable, for sfm's "NG". Shape shape_g, scale sig_u/shape_g, so E[u]=σuE[u] = \sigma_u whatever shape is chosen.

u_nak, y_pcs_nak

Nakagami one-sided error term and dependent variable, for sfm's "NNAK". Shape m_nak, spread Ω=σu2\Omega = \sigma_u^2. The default m_nak = 1 is the Rayleigh case, so this column coincides in distribution with y_pcs_r at the default.

u_ge, y_pcs_ge

Generalized-exponential one-sided error term and dependent variable, for sfm's "NGE", drawn as u=−σulog⁡(1−U)u = -\sigma_u\log(1 - \sqrt{U}) with U∼Unif(0,1)U \sim \mathrm{Unif}(0,1).

u_ln, y_pcs_ln

Lognormal one-sided error term and dependent variable, for sfm's "NLN". Log-scale mean mu_ln and log-scale standard deviation sig_u.

u_w, y_pcs_wb

Weibull one-sided error term and dependent variable, for sfm's "NW". Shape k_w, scale sig_u.

con

A constant column set to 1, potentially for use in estimation.

Author(s)

David H. Bernstein

See Also

data_gen_p for the panel generator, and sfm, zsfm and ttsfm for the estimators these columns are built for.

rnorm, runif, rt, rexp, rcauchy, rtruncnorm (if available).

Examples

# Generate 100 observations of SFA data
data_sfa <- data_gen_cs(
  N     = 100,
  rand  = 123,
  sig_u = 0.5,
  sig_v = 0.2,
  cons  = 5,
  beta1 = 1.5,
  beta2 = 2.0,
  a     = 5,   # degrees of freedom for T/Half-T
  mu    = 0.1  # mean for Truncated Normal
)

# Display the first few rows of the generated data
head(data_sfa)

# Example of a Normal-Half-Normal SFA model data
summary(data_sfa$y_pcs)
plot(density(data_sfa$y_pcs))

Generate Panel Data for Stochastic Frontier Analysis

Description

data_gen_p generates simulated panel data for estimating various panel stochastic frontier models, including the Generalized True Random Effects (GTRE), True Random Effects (TRE), Pooled Cross-Section (PCS), and True Fixed Effects (TFE) models. The function returns the data as a pdata.frame. All variants are produced so that the user can select those that they want.

Usage

data_gen_p(t, N, rand, sig_u, sig_v, sig_r, sig_h, cons, tau = 0.5,
                  mu = 0, beta1, beta2, eta = 0.1, b_k90 = 0.05, c_k90 = 0.01,
                  d_k90 = 0.05, e_k90 = -0.005, rho_mvtn = 0.5,
                  delta_fd = mu, fd_draw = c("folded", "truncated"))

Arguments

t

The number of time periods.

N

The number of individuals.

rand

A seed for the random number generator to ensure reproducibility.

sig_u

The standard deviation (σu\sigma_u) for the one-sided error component (uitu_{it}).

sig_v

The standard deviation (σv\sigma_v) for the two-sided error component (vitv_{it}).

sig_r

The standard deviation (σr\sigma_r) for the two-sided individual effect (rir_i).

sig_h

The standard deviation (σh\sigma_h) for the one-sided individual effect (hih_i).

cons

The constant term (β0\beta_0) for the frontier models.

tau

The dependence parameter (τ\tau) used for the y_tfe (TFE) model formulation, default is 0.5. See Chen, Schmidt, and Wang (2014, Journal of Econometrics).

mu

The pre-truncation mean (μ\mu) of the firm-level inefficiency ui∗u_i^* behind y_fd. Default 0. See Wang and Ho (2010, Journal of Econometrics).

delta_fd

The coefficient δ\delta in the scaling function uit=exp⁡(δzit)ui∗u_{it} = \exp(\delta z_{it}) u_i^* behind y_fd. Defaults to mu, which reproduces this function's historical output exactly; set it separately for a design in which μ\mu and δ\delta differ.

fd_draw

How ui∗u_i^* is drawn for y_fd. "folded" (the default, and the historical behaviour) takes ∣N(μ,σu2)∣|N(\mu, \sigma_u^2)|; "truncated" takes N+(μ,σu2)N^+(\mu, \sigma_u^2), which is what Wang and Ho (2010) and psfm(model_name = "FD") assume. The two coincide exactly at mu = 0 and differ otherwise. Because the truncated draw consumes random numbers differently, it also changes the columns generated after it (z_gtre, u_gtre, u_inv and those built from them), so do not mix columns from the two modes.

beta1

The coefficient for the x1 variable (β1\beta_1).

beta2

The coefficient for the x2 variable (β2\beta_2).

eta

The decay-rate parameter (η\eta) for the time-varying inefficiency component u_bc92, following Battese and Coelli (1992): uit=uiexp⁡(−η(Ti−t))u_{it} = u_i \exp(-\eta(T_i - t)). Default is 0.1. See Battese and Coelli (1992, Journal of Productivity Analysis).

b_k90, c_k90

Coefficients of the Kumbhakar (1990) inefficiency path Bit=(1+exp⁡(bt+ct2))−1B_{it} = (1 + \exp(bt + ct^2))^{-1} behind y_k1990, which targets psfm(model_name = "K1990"). The defaults give a path running from 0.485 to 0.182 over ten periods. Note that bb and cc are weakly identified: quite different pairs produce near-identical fitted paths.

rho_mvtn

Within-firm correlation of inefficiency across periods for the y_pl_mvtn column, which targets psfm(model_name = "PL80_MVTN"). The inefficiency vector is drawn from N(0,σu2[(1−ρ)I+ρ11′])N(0, \sigma_u^2[(1-\rho)I + \rho \mathbf{1}\mathbf{1}']) truncated to the negative orthant, so 0 gives inefficiency independent over time and values near 1 approach the time-invariant "PL80" case. Default 0.5. Note the realized correlation of the drawn column is ATTENUATED relative to rho by the truncation (about 0.29 at rho = 0.5, T=4T = 4) – rho parameterizes the underlying normal, not the truncated draw.

d_k90, e_k90

Coefficients of the modified Kumbhakar path Bit=1+d(t−Ti)+e(t−Ti)2B_{it} = 1 + d(t - T_i) + e(t - T_i)^2 behind y_k1990m, which targets psfm(model_name = "K1990modified"). The defaults give a path running from 0.145 to 1 over ten periods.

Details

A pdata.frame object with N×tN \times t observations, containing the following columns:

  • name Individual identifier.

  • year Time period identifier.

  • cons The constant term used in the data generation.

  • x1, x2 Explanatory variables generated from a log-uniform distribution.

  • x1_w, x2_w Explanatory variables with dependence parameter τ\tau and linkage with rir_i, used for the TFE model.

  • u, v, r, h The generated error and individual effect components.

  • y_gtre, y_tre, y_pcs, y_tfe Output variables for the Production Frontier models, including the constant.

  • y_gtre_nc, y_tre_nc, y_pcs_nc Output variables for the Production Frontier models, excluding the constant.

  • c_gtre, c_tre, c_pcs, c_tfe Output variables for the Cost Frontier models, including the constant.

  • c_gtre_nc, c_tre_nc, c_pcs_nc Output variables for the Cost Frontier models, excluding the constant.

  • y_fd Output variable for the first difference model (see Wang and Ho, 2010).

  • x_fd Explanatory variable for the y_fd model.

  • u_fd_star, z_fd, r_fd, u_fd Components used to generate y_fd.

  • u_gtre, z_gtre, zp_gtre, y_gtre_z,y_gtre_zz,y_tre_z Variables for models with heteroskedastic inefficiency (σu,i=exp⁡(0.33+0.61Zi))\sigma_{u,i} = \exp(0.33 + 0.61 Z_{i})). Variables for models with heteroskedastic persistent inefficiency (σh,i=exp⁡(0.25+0.21Zi))\sigma_{h,i} = \exp(0.25 + 0.21 Z_{i})).

  • u_inv, y_ssfe Time-invariant firm-level inefficiency (ui∼∣N(0,σu2)∣u_i \sim |N(0,\sigma_u^2)|, constant within individual across tt) and the corresponding output variable, for psfm's "SSFE" and "PL80".

  • u_bc92, y_bc92 Time-varying inefficiency following a Battese-Coelli (1992) decay structure (uit=uiexp⁡(−η(Ti−t))u_{it} = u_i \exp(-\eta(T_i - t))) and the corresponding output variable, for psfm's "BC92".

  • B_k90, u_k90, y_k1990 The Kumbhakar (1990) inefficiency path Bit=(1+exp⁡(bt+ct2))−1B_{it} = (1 + \exp(b t + c t^2))^{-1} (controlled by b_k90, c_k90), the resulting inefficiency uit=Bituiu_{it} = B_{it} u_i, and the corresponding output variable, for psfm's "K1990".

  • B_k90m, u_k90m, y_k1990m The modified Kumbhakar path Bit=1+d(t−Ti)+e(t−Ti)2B_{it} = 1 + d(t - T_i) + e(t - T_i)^2 (controlled by d_k90, e_k90), the resulting inefficiency, and the corresponding output variable, for psfm's "K1990modified".

  • h_z The heteroskedastic persistent inefficiency hih_i expanded to one value per observation (constant within individual), used to build y_gtre_zz.

The data is generated based on standard Stochastic Frontier Analysis (SFA) formulations, primarily for a **Production Frontier** where the one-sided error component uitu_{it} is subtracted:

  • y_gtre: GTRE model: yit=β0+β1x1,it+β2x2,it+ri−hi+vit−uity_{it} = \beta_0 + \beta_1 x_{1,it} + \beta_2 x_{2,it} + r_i - h_i + v_{it} - u_{it}

  • y_tre: TRE model: yit=β0+β1x1,it+β2x2,it+ri+vit−uity_{it} = \beta_0 + \beta_1 x_{1,it} + \beta_2 x_{2,it} + r_i + v_{it} - u_{it}

  • y_pcs: PCS model: yit=β0+β1x1,it+β2x2,it+vit−uity_{it} = \beta_0 + \beta_1 x_{1,it} + \beta_2 x_{2,it} + v_{it} - u_{it}

  • y_tfe: TFE model: yit=β1x1,itw+β2x2,itw+ri+vit−uity_{it} = \beta_1 x_{1,it}^w + \beta_2 x_{2,it}^w + r_i + v_{it} - u_{it}

  • y_gtre_z: GTRE with Heteroskedastic uitu_{it}: σu,i=exp⁡(0.33+0.61Zi)\sigma_{u,i} = \exp(0.33 + 0.61 Z_i).

  • y_gtre_zz: GTRE with Heteroskedastic uitu_{it}: σu,i=exp⁡(0.33+0.61Zi)\sigma_{u,i} = \exp(0.33 + 0.61 Z_i) and σh,i=exp⁡(0.25+0.21Zi)\sigma_{h,i} = \exp(0.25 + 0.21 Z_{i}).

For **Cost Frontier** models, the one-sided error component uitu_{it} is added (e.g., c_gtre).

The error terms are generated as:

  • ri∼N(0,σr2)r_i \sim N(0, \sigma_r^2) (individual two-sided effect)

  • hi∼∣N(0,σh2)∣h_i \sim |N(0, \sigma_h^2)| (individual one-sided effect)

  • vit∼N(0,σv2)v_{it} \sim N(0, \sigma_v^2) (two-sided noise)

  • uit∼∣N(0,σu2)∣u_{it} \sim |N(0, \sigma_u^2)| (one-sided inefficiency)

The First-Difference estimation model (y_fd) uses a variation where ri,fd∼U(0,1)r_{i,fd} \sim U(0,1) and uit,fd=exp⁡(δzit)ui∗u_{it,fd} = \exp(\delta z_{it}) u_i^*, with ui∗u_i^* time-invariant: folded normal by default, truncated normal under fd_draw = "truncated" (see fd_draw).

Value

A pdata.frame object containing N×tN \times t observations suitable for Stochastic Frontier Analysis (SFA).

Author(s)

David H. Bernstein

References

Chen, Y., Schmidt, P., & Wang, H. (2014). Consistent estimation of the fixed effects stochastic frontier model. Journal of Econometrics, 181(2), 65-76.

Filippini, M., & Greene, W. H. (2016). Persistent and transient productive inefficiency: a maximum simulated likelihood approach. Journal of Productivity Analysis, 45, 187-196.

Wang, H., & Ho, C. M. (2010). Estimating fixed-effect panel stochastic frontier models by model transformation. Journal of Econometrics, 157(2), 286-296.

See Also

data_gen_cs for the cross-sectional generator, and psfm for the estimators these columns are built for.

Examples

library(sfa) 
# Generate a dataset 
data_trial <- data_gen_p(t=10, N=100, rand = 100, 
                       sig_u = 1,  sig_v = 0.3, 
                       sig_r = .2, sig_h = .4, 
                       cons = 0.5, tau = 0.5,
                       mu= 0.5, beta1 = 0.5,
                       beta2 = 0.5)
 # See the first few rows 
 head(data_trial)

Density-power weights from a robust frontier fit

Description

Returns the weight each observation receives in the estimating equation of a maximum LqL_q-likelihood or minimum density-power divergence fit. The weight is the fitted density raised to the power cc, normalised so the largest is one. It is formed as exp⁡(clog⁡f)\exp(c \log f) rather than (exp⁡log⁡f)c(\exp \log f)^c, so an observation whose density underflows the natural scale still receives its own weight and keeps its place in the ordering.

Usage

density_weights(object, sigma_v = NULL, sigma_u = NULL, c = NULL,
                normalize = TRUE, log = FALSE)

Arguments

object

A fitted frontier model, or a numeric vector of composed residuals.

sigma_v, sigma_u

Scale parameters. Required when object is a numeric vector; taken from the fit otherwise.

c

Robustness tuning parameter. Taken from the fit when available.

normalize

Logical; scale so the maximum weight is one. Default TRUE.

log

Logical; return log⁡w\log w instead of ww. Default FALSE. Use this to rank or threshold observations: the weights of the most heavily discounted observations span hundreds of orders of magnitude and are not all distinguishable once exponentiated.

Value

A numeric vector of weights in (0,1](0, 1], one per observation, or of log weights in (−∞,0](-\infty, 0] when log = TRUE.

What this diagnostic can and cannot find

The weight is a function of the fitted density at the observation, so it responds to observations lying far from the fitted surface. It has no purchase on an observation that is wrong in a regressor and, because the fitted surface bends toward it, ends up close to that surface. A mis-recorded input can therefore be highly influential and still receive a weight near one. This is not a defect of the implementation: the bounded influence of these estimators is bounded with respect to response contamination, conditional on a fixed or bounded design. Pair this with a leverage diagnostic, and with case deletion, before concluding that an observation is or is not driving a result.

References

Bernstein, D.H., Parmeter, C.F. and Wright, I.A. (2026). On Robust Estimation of the Stochastic Frontier Model. Working paper.

See Also

hscore_select, calibrate_c

Examples

set.seed(1)
e <- rnorm(200, 0, 0.3) - abs(rnorm(200, 0, 0.6))
w <- density_weights(e, sigma_v = 0.3, sigma_u = 0.6, c = 0.217)
summary(w)
head(order(w), 10)   # the observations the estimator discounts most

## Rank on the log scale when the fit is sharply peaked: there the smallest
## weights are far below the smallest positive double and cannot be told
## apart once exponentiated.
lw <- density_weights(e, sigma_v = 0.02, sigma_u = 0.9, c = 0.217, log = TRUE)
head(order(lw), 10)

Technical Efficiency Scores

Description

Extracts observation-specific technical efficiency from a fitted frontier, choosing among the three usual point predictors and between the two scales the dependent variable may be on.

Usage

efficiency(object, type = c("bc", "jlms", "mode"), logDepVar = TRUE,
           newdata = NULL)

Arguments

object

An "sfareg" fit.

type

Which predictor of uiu_i to use. "bc" (the default) is Battese–Coelli, E[exp⁡(−ui)∣εi]E[\exp(-u_i) \mid \varepsilon_i]; "jlms" is Jondrow et al., exp⁡(−E[ui∣εi])\exp(-E[u_i \mid \varepsilon_i]); "mode" uses the mode of the posterior of uiu_i.

logDepVar

TRUE (the default) if the dependent variable is on a log scale, so that efficiency is exp⁡(−ui)\exp(-u_i); FALSE if it is on a level scale, giving 1−ui/f(xi)1 - u_i / f(x_i) for a production frontier and f(xi)/(f(xi)+ui)f(x_i) / (f(x_i) + u_i) for a cost frontier (inefdec = FALSE).

newdata

Data to rebuild the fitted frontier from, needed only when logDepVar = FALSE. Supply it when the data used for the fit can no longer be recovered from the stored call.

Details

Which predictor. The three differ in what they report about the same posterior, not in how well they are estimated. E[exp⁡(−u)]≥exp⁡(−E[u])E[\exp(-u)] \ge \exp(-E[u]) by Jensen's inequality, so "bc" is never below "jlms"; the gap widens as the posterior spreads out. "mode" is the only one that can equal exactly 1, and it does so for every observation whose posterior mean is negative – typically much of the efficient tail. That is a property of the predictor rather than a defect: where the posterior mean of uu is negative the single most likely value really is the boundary. It is available only for the models whose posterior is a truncated normal ("NHN", "NHN_Z", "NE", "NTN").

Which scale. The package elsewhere assumes throughout that the dependent variable is logged, which is the usual case and makes uu a proportional shortfall. logDepVar = FALSE instead forms (f(xi)−ui)/f(xi)(f(x_i) - u_i)/f(x_i) for a production frontier, and for a cost frontier, where y=f(x)+u+vy = f(x) + u + v, cost efficiency f(xi)/(f(xi)+ui)f(x_i)/(f(x_i) + u_i): minimum over actual cost, the level-scale counterpart of exp⁡(−ui)\exp(-u_i). The orientation is read from the fit's inefdec, including when it was passed as a variable. That is a ratio of two estimated quantities and is less well behaved than the log version: it is undefined where the fitted frontier is zero and negative where uu exceeds it, both of which happen near zero output. A warning is issued if any score comes back negative or non-finite, since silence there would be the real failure.

Value

A numeric vector of efficiency scores, one per observation used in the fit.

See Also

efficiency_ci for interval estimates of the same quantity.

Examples

set.seed(2)
n  <- 400
x1 <- rnorm(n); x2 <- rnorm(n)
y  <- 6 + 0.5 * x1 + 0.5 * x2 + rnorm(n, 0, 0.3) - abs(rnorm(n, 0, 0.8))
d  <- data.frame(y = y, x1 = x1, x2 = x2)

fit <- sfm(y ~ x1 + x2, model_name = "NHN", data = d)

summary(efficiency(fit))                      # Battese-Coelli, log scale
summary(efficiency(fit, type = "jlms"))       # never above the above
summary(efficiency(fit, type = "mode"))       # reaches exactly 1
summary(efficiency(fit, logDepVar = FALSE, newdata = d))

Confidence Intervals for Individual Inefficiency and Efficiency

Description

Horrace and Schmidt (1996) intervals for the inefficiency term uiu_i of a fitted stochastic frontier model, and the implied intervals for technical efficiency exp⁡(−ui)\exp(-u_i).

Usage

efficiency_ci(object, level = 0.95, type = c("both", "u", "te"))

Arguments

object

an object of class "sfareg", as returned by sfm.

level

coverage of the interval, a single number strictly between 0 and 1. Defaults to 0.95.

type

which intervals to return: "both" (the default), "u" for inefficiency only, or "te" for technical efficiency only.

Details

Conditional on the fitted parameters, ui∣eiu_i \mid e_i is normal with mean mu_star and standard deviation sigma_star, truncated below at zero – the same posterior the Jondrow et al. (1982) and Battese and Coelli (1988) point predictors average over. Inverting it at (1−α)/2(1-\alpha)/2 and 1−(1−α)/21-(1-\alpha)/2 gives the bounds in closed form, so the interval costs no estimation beyond the fit itself. The efficiency bounds follow by monotonicity, with the endpoints swapped, since exp⁡(−u)\exp(-u) is decreasing in uu.

These intervals condition on the estimated parameters. They describe where uiu_i sits given eie_i and a known frontier, and make no allowance for sampling error in the slopes or the variance parameters. Horrace and Schmidt are explicit about this, and it is why the intervals do not narrow as the sample grows: they measure the irreducible difficulty of splitting a single residual into noise and inefficiency, not estimation uncertainty. Read a wide interval as a warning against reading much into that unit's rank.

Available for the models whose posterior really is a truncated normal: "NHN", "NHN_Z", "NE" and "NTN". Every other model_name has a posterior of a different shape, for which these formulas do not hold; the function reports that rather than returning a misleading number.

Value

A data frame with one row per observation, carrying u_lower, u_hat, u_upper and/or te_lower, te_hat, te_upper according to type, and a "level" attribute. u_hat is the Jondrow et al. (1982) posterior mean and te_hat the Battese and Coelli (1988) score – exactly the values the fit already reports, taken from the fitted object rather than recomputed.

References

Horrace, W. C. and Schmidt, P. (1996). Confidence statements for efficiency estimates from stochastic frontier models. Journal of Productivity Analysis, 7, 257–282.

Jondrow, J., Lovell, C. A. K., Materov, I. S. and Schmidt, P. (1982). On the estimation of technical inefficiency in the stochastic frontier production function model. Journal of Econometrics, 19, 233–238.

Battese, G. E. and Coelli, T. J. (1988). Prediction of firm-level technical efficiencies with a generalized frontier production function and panel data. Journal of Econometrics, 38, 387–399.

See Also

sfm, sfa_diagnostics

Examples

dat <- data_gen_cs(N = 200, rand = 1, sig_u = 1, sig_v = 0.3,
                   cons = 0.5, beta1 = 0.5, beta2 = 0.5, a = 1, mu = 0.5)
fit <- sfm(y_pcs ~ x1 + x2, data = dat, model_name = "NHN")

ci <- efficiency_ci(fit, level = 0.90)
head(ci)

## How much of an efficiency ranking is real? Compare the spread of the point
## predictions against the width of a single unit's interval.
diff(range(ci$te_hat))
median(ci$te_upper - ci$te_lower)

Were the regressors endogenous after all?

Description

A Wald test of H0 ⁣:ρ=0H_0\!: \rho = 0 on an ivsfm fit – the hypothesis that the noise vv is uncorrelated with the reduced-form errors of the endogenous regressors, so that the endogeneity correction was not needed and a plain sfm fit would have been consistent. This is the Durbin–Wu–Hausman idea carried into the stochastic frontier, and is Equation (25) of Hou, Ramalho and Roseta-Palma (2025).

Usage

endogeneity_test(object, level = 0.05)

Arguments

object

A fit from ivsfm with model_name = "IVLIML" or "IVCF". "C2SLS" is a moment estimator and carries no correction parameter, so it is refused.

level

Size used for the reported verdict.

Details

The statistic. With ρ^\hat\rho the vector of correlations between vv and the reduced-form errors,

W=ρ^′ [Var^(ρ^)]−1 ρ^  ∼  χ2(q),W = \hat\rho' \,[\widehat{\mathrm{Var}}(\hat\rho)]^{-1}\, \hat\rho \;\sim\; \chi^2(q),

qq the number of endogenous regressors. With a single endogenous regressor it is exactly the squared z-ratio already printed for rho_ in the fit.

Why this is an ordinary chi-square. ρ\rho lives in the open unit ball, so ρ=0\rho = 0 is an interior point of the parameter space. That is the opposite of the situation inefficiency_test faces, where σu=0\sigma_u = 0 sits on a boundary and the null distribution is a chi-bar-square mixture. No mixture is needed here.

Standard errors for ρ\rho. ivsfm parameterises the correlations as ρ=t/1+t′t\rho = t/\sqrt{1 + t't}, which keeps them inside the unit ball for any real tt. Before version 1.2.0 the standard errors of ρ\rho were reported as NA on the ground that this map has a non-diagonal Jacobian. It does, but the Jacobian is short –

∂ρi/∂tj=δij/s−titj/s3,s=1+t′t,\partial \rho_i/\partial t_j = \delta_{ij}/s - t_i t_j / s^3, \qquad s = \sqrt{1 + t't},

that is J=(I−ρρ′)/sJ = (I - \rho\rho')/s – so Var(ρ^)=J Var(t^) J′\mathrm{Var}(\hat\rho) = J\,\mathrm{Var}(\hat t)\,J' by the delta method. At the null J=IJ = I, which is what makes the test well calibrated there. ivsfm now reports these standard errors and carries the full Var(ρ^)\mathrm{Var}(\hat\rho) as $vcov_rho, which is what the joint test needs.

Size and power. Measured in this package over 1,000 replications per cell on the design of Hou, Ramalho and Roseta-Palma's section 4.1.1, with T=20T = 20 and NN varying. The ρ=0\rho = 0 block is size; the rest is power. Monte Carlo standard errors at 1,000 replications are 0.0095, 0.0069 and 0.0031 at the three levels.

ρ\rho nn 10% 5% 1% se/sdse/sd
0.0 500 0.101 0.051 0.014 0.999
0.0 1000 0.098 0.051 0.006 1.008
0.0 2000 0.100 0.053 0.009 1.009
0.0 4000 0.092 0.043 0.004 1.045
0.1 500 0.422 0.298 0.116 1.027
0.1 1000 0.629 0.492 0.270 0.991
0.1 2000 0.860 0.787 0.578 0.983
0.1 4000 0.984 0.969 0.897 0.970
0.2 500 0.879 0.807 0.611 0.976
0.2 1000 0.986 0.976 0.902 0.951
0.2 2000 1.000 1.000 0.998 0.971
0.2 4000 1.000 1.000 1.000 0.974

The test holds its size at every sample size tried, and power rises monotonically to one at both alternatives. The last column is the diagnostic behind that: it is the mean reported standard error of ρ^\hat\rho divided by the actual sampling standard deviation of ρ^\hat\rho across replications. The statistic is ρ^2/Var^(ρ^)\hat\rho^2/\widehat{\mathrm{Var}}(\hat\rho), so a rejection rate away from nominal comes either from ρ^\hat\rho itself or from the variance, and only the second would be a defect here. At 0.95 to 1.05 throughout, the variance estimate is sound – which is what separates this test from uhet_test, where the same ratio is 0.93 and the test is conservative in consequence.

At the paper's own design (N=100N = 100, T=20T = 20) the larger alternatives it reports are saturated: power is 1.000 at every level at both ρ=0.4\rho = 0.4 and ρ=0.8\rho = 0.8. The ρ=0.1\rho = 0.1 and 0.20.2 rows are included precisely because a column of ones says nothing about how power builds. Replication scripts: spec_tests/hou2025_table2.R and spec_tests/hou2025_endog_size.R.

Reading a non-rejection. Failing to reject is not evidence for exogeneity. ρ\rho is identified through the instruments, so weak instruments make ρ^\hat\rho imprecise and the test powerless; the print method says so rather than letting the p-value speak alone.

Value

An object of class c("sfa_endog_test", "htest") carrying statistic (the Wald WW), parameter (its degrees of freedom, the number of endogenous regressors), p.value, the estimated rho with its standard errors and z-ratios, and reject.

References

Hou, Z., Ramalho, J. J. S. and Roseta-Palma, C. (2025). Dealing with endogeneity in stochastic frontier models: A comparative assessment of estimators. Energy Economics 151, 108922.

Amsler, C., Prokhorov, A. and Schmidt, P. (2016). Endogeneity in stochastic frontier models. Journal of Econometrics 190(2), 280-288.

Kutlu, L. (2010). Battese-Coelli estimator with endogenous regressors. Economics Letters 109(2), 79-81.

See Also

ivsfm for the fit, inefficiency_test for the boundary-null case, spec_test and moment_range for the distributional assumptions.

Examples

set.seed(9)
n  <- 600
z  <- rnorm(n); x1 <- rnorm(n); eps <- rnorm(n)
x2 <- z + eps
v  <- 0.6 * eps + sqrt(1 - 0.36) * rnorm(n)
d  <- data.frame(y = 0.5 * x1 + 0.5 * x2 + v - abs(rnorm(n)),
                 x1 = x1, x2 = x2, z = z)

fit <- ivsfm(y ~ x1 + x2, endogenous = ~ x2, instruments = ~ z,
             data = d, model_name = "IVCF")
endogeneity_test(fit)

Stochastic frontier tolerating the wrong skewness

Description

Fits the extended stochastic frontier model of Hafner, Manner and Simar (2018), in which one parameter γ\gamma carries both the scale of inefficiency (its size) and the direction of skewness (its sign), so a wrongly skewed sample gives a well-defined non-degenerate fit instead of collapsing.

Usage

esfm(formula, data, dist = c("halfnormal", "exponential"), inefdec = TRUE,
     start = NULL)

symmetry_test(object)

Arguments

formula, data

Model and data, as for sfm.

dist

Base density for the inefficiency term.

inefdec

TRUE for a production frontier, FALSE for cost.

start

Optional numeric start, c(gamma, sigma_v, beta).

object

An "esfm" fit.

Details

The classical frontier implies a negatively skewed composed error. When the sample skewness comes out positive – common at small nn or large σv/σu\sigma_v/\sigma_u, and a small-sample accident rather than evidence against the model – the MLE collapses: σu\sigma_u is exactly zero, every firm is fully efficient, and the fit is uninformative. That is the Type I failure skewness_test reports.

This model keeps the frontier and widens the distribution of uu. For γ>0\gamma > 0, uu has the classical density with scale ∣γ∣|\gamma|; for γ<0\gamma < 0 that density is mirrored about zero, shifted right by B=a0∣γ∣B = a_0|\gamma| and truncated to [0,B][0, B], giving negative skewness for uu and so positive skewness for the composed error. The truncation constant a0a_0 is fixed, not estimated: it is chosen so E[u]=k1∣γ∣E[u] = k_1|\gamma| whichever sign γ\gamma takes, which keeps the model a one-parameter scale family and avoids the identification difficulties of bounded-inefficiency models. a0=1.3892032925a_0 = 1.3892032925 for the half-normal and 1.59362426001.5936242600 for the exponential.

The classical model is nested at γ>0\gamma > 0, and γ=0\gamma = 0 is an interior point rather than a boundary. So the likelihood ratio test of H0:γ=0H_0: \gamma = 0 – no inefficiency, symmetric errors – is an ordinary χ2(1)\chi^2(1), not the chi-bar-square mixture that inefficiency_test needs for the classical model.

What it does on a wrongly skewed sample. On a 50-observation draw with positive residual skewness, sfm(model_name = "NHN") returns σu=0.0003\sigma_u = 0.0003 and mean efficiency 0.99976 – every firm on the frontier. esfm() returns γ^=−0.57\hat\gamma = -0.57 with slopes still near their true values.

Size of the LR test, 2000 replications against the paper's 100,000, on its design (σv=1\sigma_v = 1, γ=0\gamma = 0):

nn 50 100 200 500 1000
this package, 5% 0.178 0.082 0.054 0.061 0.058
paper, 5% 0.152 0.076 0.061 0.054 0.053

The test is badly oversized below n=100n = 100 and usable from n=200n = 200, which is the paper's own conclusion.

Efficiency bias, and a caveat. The paper reports smaller bias in mean efficiency than the classical model even when the population skewness has the correct sign. That reproduces when inefficiency is not small: at γ=0.5\gamma = 0.5, σv=0.25\sigma_v = 0.25 the bias in E[exp⁡(−U)]E[\exp(-U)] is -0.0001 at n=100n = 100 against the classical model's +0.021. But at γ=0.3\gamma = 0.3, where 22% of samples are wrongly skewed, this model overshoots the other way: bias -0.040 against the classical +0.036, and it is the larger of the two at n=50n = 50 (-0.080 against +0.044). A small true γ\gamma lets the extended model fit sizeable inefficiency of either sign to noise. It is the better choice when the wrong skewness is the problem, not a free improvement everywhere.

Value

esfm() returns an object of class "esfm" (and "sfareg"). The components most often needed:

out matrix of par, st_err, t-val for gamma, sigv and the frontier coefficients
gamma, sigma_v the two scale estimates; gamma < 0 means a wrongly skewed fit
Efficiency E[exp⁡(−U)∣W]E[\exp(-U) \mid W] for each firm
logLik, logLik_restricted log-likelihood, and that of the γ=0\gamma = 0 (OLS) fit
lr_symmetry, p_symmetry the LR statistic for γ=0\gamma = 0 and its χ2(1)\chi^2(1) p-value
wrong_skew_sample, residual_m3 whether the OLS residuals are wrongly skewed, and their third moment
model_name "EXT-HALFNORMAL" or "EXT-EXPONENTIAL"

symmetry_test() returns a one-row data frame with columns test, statistic, df (always 1), null ("chi2(1)") and p.value, read from the fit's lr_symmetry and p_symmetry.

References

Hafner, C. M., Manner, H. and Simar, L. (2018). The “wrong skewness” problem in stochastic frontier models: A new approach. Econometric Reviews 37(4), 380–400.

Almanidis, P., Qian, J. and Sickles, R. C. (2014). Stochastic frontier models with bounded inefficiency. In Festschrift in Honor of Peter Schmidt.

See Also

skewness_test, inefficiency_test, sfm

Examples

set.seed(3)
n <- 200
lx1 <- rnorm(n, 1.5, 0.3); lx2 <- rnorm(n, 1.8, 0.3)
y <- 0.9 + 0.6 * lx1 + 0.5 * lx2 + rnorm(n, 0, 0.25) - abs(rnorm(n, 0, 0.5))
d <- data.frame(y, lx1, lx2)

f <- esfm(y ~ lx1 + lx2, data = d)
f
symmetry_test(f)

Expenditure and output of Finnish electricity distribution firms

Description

Cross-sectional data on Finnish electricity distribution firms, including annual averages of expenditure and output measures over a four-year regulatory period.

Usage

data("FinnishElec")

Format

A data frame with 89 observations on the following 6 variables.

id

a character vector containing a unique identifier for each distribution firm

x

a numeric vector containing total expenditure (TOTEX*) (1000 Euros)

y1

a numeric vector containing weighted energy transmitted through the network (GWh of 0.4 kV equivalents)

y2

a numeric vector containing total length of the network (km)

y3

a numeric vector containing total number of customers connected to the network

z

a numeric vector containing the proportion of underground cables in the total network length.

Details

*TOTEX includes capital expenditure (CAPEX), controllable operational expenditure (OPEX), and estimated external cost of interruptions.

Source

Kuosmanen, T. (2012). 'Stochastic semi-nonparametric frontier estimation of electricity distribution networks: Application of the StoNED method in the Finnish regulatory model.' Energy Economics, 34(6), pp. 2189-2199. doi:10.1016/j.eneco.2012.03.005

Examples

data(FinnishElec)
plot(FinnishElec)

Goodness-of-fit test for the assumed inefficiency distribution

Description

Tests the distributional assumption on uu, holding the normality of vv fixed, following Wang, Amsler and Schmidt (2011).

Usage

gof_test(object, data = NULL, test = c("ks", "chisq"),
         null = c("bootstrap", "asymptotic"), B = 199, cells = 10,
         seed = NULL)

Arguments

object

An "sfareg" fit from sfm.

data

The data the model was fitted to. sfm() does not retain it, so it must be supplied or be findable from the fit's call.

test

"ks", "chisq", or both.

null

"bootstrap" for the parametric bootstrap of Wang, Amsler and Schmidt, or "asymptotic".

B

Bootstrap replications. Each one refits the model.

cells

Number of equiprobable cells for the chi-square test.

seed

Optional seed for the bootstrap.

Details

The distribution of uu is the assumption in this model least often defended and least often tested, and it is testable: with the noise distribution maintained, the assumed uu implies a distribution for the composed error, so rejecting that is rejecting the assumed uu.

The test is on ε\varepsilon, not on u^\hat u. Wang, Amsler and Schmidt point out that u^=E[u∣ε]\hat u = E[u \mid \varepsilon] is a monotonic function of ε\varepsilon, so the KS test is identical either way and the chi-square test is identical when the cells are defined conformably – but ε\varepsilon is much easier to work with. It is a mistake, not a diagnostic, to compare the observed spread of u^\hat u with the assumed density of uu: those are different distributions.

Parameter estimation cannot be ignored. Both statistics are evaluated at θ^\hat\theta, which changes their null distributions. The default "bootstrap" handles this by copying the estimation step exactly: each replication draws a composed error from the fitted model, rebuilds the response, refits, forms its own residuals, and recomputes the statistic. "asymptotic" returns a χ2(k−1−m)\chi^2(k-1-m) p-value for the chi-square statistic, which the authors note is conservative when evaluated at the MLE – that reference distribution belongs to the minimum-chi-square estimator. There is no asymptotic KS p-value here: with θ\theta estimated the Kolmogorov distribution does not apply, and Bai's (2003) martingale transformation is not implemented, so NA is returned rather than a number that would look valid.

Size and power. Measured in this package over 200 replications with B = 49, half-normal data fitted as half-normal for size and exponential data fitted as half-normal for power:

nn KS .10 KS .05 chi2 .10 chi2 .05
size 200 0.065 0.025 0.085 0.065
size 800 0.070 0.040 0.065 0.010
power 200 0.505 0.395 0.235 0.145
power 800 0.985 0.965 0.810 0.655
power 2000 1.000 1.000 0.990 0.985

Both tests hold their size, mildly conservatively. The KS test dominates the chi-square test at every sample size, which is the paper's own conclusion and the reason it is the better default: the chi-square statistic discards the within-cell information that KS uses. Power against the half-normal/exponential pair – one of the harder distinctions, since both are one-parameter and similarly shaped – is about 0.4 at n=200n = 200 and essentially 1 by n=800n = 800, so a failure to reject in a small sample is weak evidence.

Value

A data frame with one row per test: test, statistic, null, p.value and B (replications actually used).

References

Wang, W. S., Amsler, C. and Schmidt, P. (2011). Goodness of fit tests in stochastic frontier models. Journal of Productivity Analysis 35, 95–118.

Bai, J. (2003). Testing parametric conditional distributions of dynamic models. Review of Economics and Statistics 85, 531–549.

See Also

spec_test, pcomposed_model, sfm

Examples

d <- as.data.frame(data_gen_cs(N = 200, rand = 5, sig_u = 1, sig_v = 0.5,
                               cons = 0.5, beta1 = 0.5, beta2 = 0.5,
                               a = 5, mu = 0.1))
f <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d)

## Every bootstrap replication REFITS, so keep B small in an example; use
## B = 999 for a real test.
gof_test(f, data = d, B = 39, seed = 1)

gof_test(f, data = d, test = "chisq", null = "asymptotic")

Hyvarinen score for a Normal–Half-Normal frontier fit

Description

Evaluates the Hyvarinen score used to choose the robustness tuning parameter of the maximum LqL_q-likelihood and minimum density-power divergence estimators, following Sugasawa and Yonekura (2021). Lower is better.

Usage

hscore(e, sigma_v, sigma_u, c, stable = TRUE)

Arguments

e

Numeric vector of composed residuals y−x′βy - x'\beta.

sigma_v, sigma_u

Positive scalars, the noise and inefficiency scale parameters.

c

Robustness tuning parameter, c=1−qc = 1-q for MLqE and c=αc = \alpha for MDPD. c = 0 is the maximum likelihood endpoint.

stable

Logical. TRUE (default) evaluates in log space; FALSE uses the natural-scale expression.

Details

The score is

H(c)=n−1∑i{2D′′(ei)+D′(ei)2},D=(fc−1)/c,H(c) = n^{-1}\sum_i \{2 D''(e_i) + D'(e_i)^2\},\qquad D = (f^c-1)/c,

with derivatives taken with respect to the composed residual.

The score is evaluated in logarithms. The natural-scale expression contains the fitted density raised to the power c−2c-2; since c<1c<1 that exponent is close to −2-2, so a single observation whose density underflows makes the score non-finite. The failure is silent and directional: the tuning candidates that remain evaluable are those closest to maximum likelihood, so a naive implementation reports too little robustness. Set stable = FALSE to reproduce that behaviour for comparison.

Value

A single numeric value, the mean observationwise score.

References

Hyvarinen, A. (2005). Estimation of non-normalized statistical models by score matching. Journal of Machine Learning Research 6, 695–709.

Sugasawa, S. and Yonekura, S. (2021). On selection criteria for the tuning parameter in robust divergence. Entropy 23(9), 1147.

Bernstein, D.H., Parmeter, C.F. and Wright, I.A. (2026). On Robust Estimation of the Stochastic Frontier Model. Working paper.

See Also

hscore_select, calibrate_c, density_weights

Examples

set.seed(1)
e <- rnorm(500, 0, 0.3) - abs(rnorm(500, 0, 0.6))
hscore(e, sigma_v = 0.3, sigma_u = 0.6, c = 0.20)

## the two evaluations agree wherever both are computable
hscore(e, 0.3, 0.6, 0.20) - hscore(e, 0.3, 0.6, 0.20, stable = FALSE)

## but only one of them survives an observation in the far tail
e2 <- c(e, -60)
hscore(e2, 0.3, 0.6, 0.15)                  # finite
hscore(e2, 0.3, 0.6, 0.15, stable = FALSE)  # not finite

Select the robustness tuning parameter by Hyvarinen score

Description

Refits a Normal–Half-Normal stochastic frontier at each candidate value of the robustness tuning parameter and returns the value minimising the Hyvarinen score of hscore. The search is coarse-to-fine and includes the maximum-likelihood endpoint c=0c=0, so the criterion is free to decline robustification altogether.

Usage

hscore_select(object, method = c("mlqe", "psi", "mdpd"),
              range = c(0, 0.60), coarse = 0.01, fine = 0.001,
              half_width = 0.02, multistart = TRUE, verbose = FALSE)

Arguments

object

A fitted model from sfm with model_name = "NHN", used for the data, the formula and the reference starting values.

method

Robust criterion. "psi" and "mdpd" are the same estimator at a common tuning value.

range

Numeric length-2 search interval.

coarse, fine

Grid steps for the two passes.

half_width

Half-width of the refinement window around the coarse minimum.

multistart

Logical; see Details.

verbose

Logical, print progress.

Details

Two failure modes are guarded here, both of which return a converged fit and a finite criterion value if they are not.

Silent grid loss. The criterion is evaluated in log space (see hscore). Evaluated naively it is non-finite wherever any fitted density underflows, which discards candidates non-randomly – those nearest maximum likelihood survive – and so understates the robustness the data call for.

Warm-start capture. Walking the grid upward and warm-starting each fit from the previous one is efficient but unguarded: if one fit falls into the degenerate σu→0\sigma_u \to 0 basin, every later fit inherits it. With multistart = TRUE each candidate is fitted from both the warm start and a fixed reference start, keeping the better objective.

Value

An object of class "sfa_hscore": the selected tuning parameter c, its criterion value hscore, the whole search path as a data frame, the refitted model fit, and n_finite, how many candidates were scorable out of how many tried.

Reading the result

A flat region around the minimum is not non-identification in the usual sense; it means the criterion discriminates weakly among nearby tuning values. The drop from c=0c=0 to the minimum is a descriptive score difference, not a test of maximum likelihood against a robust alternative. Where the selected value matters, compare it with the fixed calibration from calibrate_c and check its stability by resampling: the selection can be bimodal across resamples even when the full-sample criterion clearly prefers a robust solution.

References

Sugasawa, S. and Yonekura, S. (2021). On selection criteria for the tuning parameter in robust divergence. Entropy 23(9), 1147.

Bernstein, D.H., Parmeter, C.F. and Wright, I.A. (2026). On Robust Estimation of the Stochastic Frontier Model. Working paper.

See Also

hscore, calibrate_c, density_weights, sfm

Examples

set.seed(3)
n <- 200
x <- runif(n, 1, 10)
y <- 1 + 0.5 * log(x) + rnorm(n, 0, 0.3) - abs(rnorm(n, 0, 0.6))
dat <- data.frame(y = y, x = log(x))
fit <- sfm(y ~ x, data = dat, model_name = "NHN")

sel <- hscore_select(fit, method = "mlqe")
sel
plot(sel$path$c, sel$path$hscore, type = "l",
     xlab = "c", ylab = "Hyvarinen score")
abline(v = sel$c, lty = 3)

Paddy farming in Aurepalle, India, 1975-76 to 1984-85

Description

Panel data on 34 paddy farmers from Aurepalle, India, collected over ten years (1975-76 to 1984-85). The panel is unbalanced: farmers are observed between 2 and 10 times. Includes farmer characteristics (age, schooling) and production variables (output, land, labor, bullocks, input costs).

Usage

data("Indian")

Format

A data frame with 273 observations (an unbalanced panel of 34 farmers over 10 years) on the following 10 variables.

id

a numeric vector containing a unique identifier for each farmer

yr

a numeric vector containing the year of the observation

age

a numeric vector containing the age of the primary decision maker

school

a numeric vector containing the number of years of schooling of the primary decision maker

yvar

a numeric vector containing the natural logarithm of the total value of output (rupees)

Lland

a numeric vector containing the natural logarithm of the total area of land operated (ha)

PIland

a numeric vector containing the proportion of land that is irrigated

Llabor

a numeric vector containing the natural logarithm of the total number of hours of hired and family labour used

Lbull

a numeric vector containing the natural logarithm of the number of hours of bullock labour used

Lcost

a numeric vector containing the natural logarithm of the value of inputs including fertilizer, manure, pesticides, machinery, etc.

Source

Battese, G.E. and Coelli, T.J. (1995) 'A model for technical inefficiency effects in a stochastic frontier production function for panel data', Empirical Economics, 20(2), pp. 325-332. doi:10.1007/BF01205442.

References

Battese, G.E. and Coelli, T.J. (1992) 'Frontier production functions, technical efficiency and panel data: With application to paddy farmers in India', Journal of Productivity Analysis, 3(1-2), pp. 153-169. doi:10.1007/BF00158774.

Examples

data(Indian)

Test whether there is any technical inefficiency

Description

Tests H0H_0: no technical inefficiency, i.e. γ=0\gamma = 0 in the Battese-Corra parameterization γ=σu2/(σu2+σv2)\gamma = \sigma_u^2/(\sigma_u^2 + \sigma_v^2), following Section 3 of Coelli (1995).

Usage

inefficiency_test(object, test = c("lr_1sided", "lr", "wald", "m3t"),
                  level = 0.05)

Arguments

object

An "sfareg" fit from sfm, with model_name = "NHN" or "NTN".

test

Which statistics to compute; any subset, all by default.

level

Significance level used for the reject column.

Details

The two tests most often reported are the two with the wrong size. H0H_0 puts γ\gamma on the boundary of the parameter space, so the usual asymptotics fail. The likelihood ratio statistic is not χ2(1)\chi^2(1) but the Gourieroux, Holly and Monfort (1982) mixture 12χ2(0)+12χ2(1)\tfrac12\chi^2(0) + \tfrac12\chi^2(1), whose 5% critical value is 2.71 rather than 3.84; using the χ2(1)\chi^2(1) value makes the naive LR test conservative. The Wald ratio γ^/sγ^\hat\gamma/s_{\hat\gamma} is worse: it is not asymptotically standard normal at the boundary at all.

Measured in this package on 400 replications per cell, half-normal data with σu=0\sigma_u = 0 and a nominal 5% level:

nn LR one-sided LR naive Wald M3T
100 0.060 0.030 0.230 0.055
200 0.060 0.033 0.203 0.050
400 0.055 0.033 0.217 0.045
800 0.060 0.035 0.182 0.062

The one-sided LR test and the third-moment test hold their size; the naive LR test rejects about half as often as it should; the Wald test rejects three to four times too often and does not improve with nn. Under the alternative (σu=σv=1\sigma_u = \sigma_v = 1) the one-sided LR test has the better power of the two valid tests at every sample size, which is Coelli's recommendation. The Wald column's apparently high power is not comparable, because its size is wrong.

For model_name = "NTN" the null restricts both γ\gamma and μ\mu, and the mixture becomes 14χ2(0)+12χ2(1)+14χ2(2)\tfrac14\chi^2(0) + \tfrac12\chi^2(1) + \tfrac14\chi^2(2).

The "m3t" statistic is the same one skewness_test reports as test = "coelli"; it is repeated here so the four tests can be compared in a single table.

Value

A data frame with one row per test: test, statistic, null (the reference distribution), p.value and reject. The log-likelihoods under both hypotheses are attached as attributes.

References

Coelli, T. (1995). Estimators and hypothesis tests for a stochastic frontier function: A Monte Carlo analysis. Journal of Productivity Analysis 6, 247–268.

Gourieroux, C., Holly, A. and Monfort, A. (1982). Likelihood ratio test, Wald test, and Kuhn-Tucker test in linear models with inequality constraints on the regression parameters. Econometrica 50, 63–80.

See Also

skewness_test, spec_test, sfm

Examples

d <- data_gen_cs(N = 300, rand = 7, sig_u = 1, sig_v = 0.5, cons = 0.5,
                 beta1 = 0.5, beta2 = 0.5, a = 5, mu = 0.1)
f <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = as.data.frame(d))
inefficiency_test(f)

Which observations are driving this frontier fit?

Description

Empirical influence function and case-influence measures for a fitted stochastic frontier. Answers which observations move the estimates, and whether the specification lets any single observation move them without bound.

Usage

influence_sfa(object, scale = TRUE)

Arguments

object

An "sfareg" fit. It must retain its likelihood, so fit with keep_objective = TRUE; the score matrix cannot otherwise be built. For psfm panel fits ("GTRE" (estimator = "sml"), "TRE", "GTRE_Z", "TRE_Z", "GTRE_FML", "TFE_WMLE", "FD", "PL80", "PL80_MVTN", "BC92", "K1990" and "K1990modified") the rows are firms, not observations, so the diagnostics measure the influence of deleting a whole firm. From 1.2.1 zsfm, lcsfm, ttsfm ("TTNE", "TTHN"), copsfm, ivsfm ("IVLIML", "IVCF") and selsfm can supply scores too; for those the influence is reported on the scale coef uses, which is not the scale they estimate on. For selsfm the rows are the selected observations only.

scale

Multiply the influence function by nn, so that influence[i, ] approximates the effect on the estimate of deleting observation ii and is therefore on the same scale as the coefficients. Default TRUE.

Details

The standard outlier rules do not transfer to this model. Least-median-of-squares and its relatives ignore the asymmetry of the composed error, so a genuinely large uiu_i – an inefficient firm, the thing being measured – reads as an outlier and is discarded, which defeats the analysis.

The influence function is the right object instead. Maximum likelihood is an M-estimator, so its influence function is a linear transformation of the score, IFi∝I(θ)−1ψ(yi,θ)IF_i \propto I(\theta)^{-1}\psi(y_i,\theta), and the estimator is B-robust exactly when that is bounded. Both pieces are already available: estfun supplies the per-observation scores and vcov the inverse information.

Which sensitivity to read. The raw sup-norm depends on how the parameters happen to be scaled, so it is not comparable between models with different parameter vectors. On a clean sample of 200 it reads 72.6 for "NHN" and 1322.5 for "tHN", which appears to say the robust specification is far worse; all it reflects is tHN's extra ν\nu on its own scale. sensitivity_std measures the same influence function in the information metric and is invariant to reparameterisation. On that scale, contaminating a single response takes "NHN" from 13.5 to 32.3 and "tHN" from 13.1 to 17.2 – Stead, Wheat and Greene's finding, and invisible in the raw number.

Neither number has an absolute threshold. The comparison to make is between specifications on the same data, not either number against a cut-off.

What to do about a large value. Nothing, by deletion. A large sensitivity is a property of the specification: under an unbounded influence function the next most extreme point simply replaces the one removed. The remedies are a specification with bounded influence – Stead, Wheat and Greene's Student's tt noise term, model_name = "tHN" – or a robust divergence criterion, sfm(robust = ), whose tuning is chosen by hscore_select or calibrate_c.

What this cannot see. Like density_weights, it is computed at the fitted surface. An observation mis-recorded in a regressor can bend that surface toward itself and so appear unremarkable here. Pair this with a leverage check before concluding that an observation is not driving a result.

Value

An object of class "sfa_influence":

influence

n×pn \times p matrix, the empirical influence function.

d

Length-nn case influence n si′Vsin\,s_i'Vs_i, the same quadratic form as Cook's distance and invariant to reparameterisation.

sensitivity_std

sup⁡idi\sup_i \sqrt{d_i}, the self-standardised sensitivity – the one to compare across models.

sensitivity

sup⁡i∥IFi∥\sup_i \|IF_i\|, the raw sup-norm.

max_abs_by_parameter

Largest ∣IF∣|IF| in each coordinate.

References

Stead, A.D., Wheat, P. and Greene, W.H. (2023). Robust maximum likelihood estimation of stochastic frontier models. European Journal of Operational Research, 309(1), 188–201.

Bernstein, D.H., Parmeter, C.F. and Wright, I.A. (2026). On Robust Estimation of the Stochastic Frontier Model. Working paper.

See Also

density_weights, hscore_select, sfm, sfareg-methods

Examples

set.seed(3)
n <- 120
x <- runif(n, 1, 10)
d <- data.frame(y = 1 + 0.5 * log(x) + rnorm(n, 0, 0.3) - abs(rnorm(n, 0, 0.6)),
                x = log(x))

fit <- sfm(y ~ x, data = d, model_name = "NHN", keep_objective = TRUE)
influence_sfa(fit)

## Contaminate one response and compare specifications on the
## self-standardised scale.
d2 <- d; d2$y[1] <- d2$y[1] - 5
a <- sfm(y ~ x, data = d2, model_name = "NHN", keep_objective = TRUE)
b <- sfm(y ~ x, data = d2, model_name = "tHN", keep_objective = TRUE)
c(NHN = influence_sfa(a)$sensitivity_std,
  tHN = influence_sfa(b)$sensitivity_std)

Stochastic Frontier Models with Endogenous Regressors

Description

Fits the stochastic frontier model of Amsler, Prokhorov and Schmidt (2016) in which one or more regressors is endogenous: correlated with the noise vv, not with the inefficiency. With uhet the model becomes that of Amsler, Prokhorov and Schmidt (2017), in which environmental variables entering the inefficiency scale may themselves be endogenous. Three estimators are provided:

model_name estimator when to use it
"IVLIML" (default) full-information maximum likelihood efficient, and its standard errors are correct as reported
"IVCF" two-step control function (Kutlu, 2010) cheaper; its standard errors are too small when the endogeneity is real
"C2SLS" corrected 2SLS closed form, no likelihood; no uhet

Usage

ivsfm(formula, endogenous, instruments, data,
      model_name = c("IVLIML", "IVCF", "C2SLS"), uhet = NULL,
      inefdec = TRUE, maxit.bobyqa = 10000, maxit.psoptim = 1000,
      maxit.optim = 1000, start_val = FALSE, PSopt = FALSE,
      optHessian = TRUE, Method = "L-BFGS-B", verbose = FALSE,
      keep_objective = FALSE, rand.psoptim = NULL)

Arguments

formula

The structural frontier, y ~ x1 + x2, listing all regressors including the endogenous ones.

endogenous

One-sided formula naming the endogenous variables, ~ x2. Each must appear in formula or in uhet.

instruments

One-sided formula of the excluded instruments, ~ w1 + w2. The included exogenous regressors are added automatically, so only the outside instruments are listed here. There must be at least as many as there are endogenous variables.

data

A data.frame holding every variable in the four formulas.

model_name

"IVLIML" (the default), "IVCF" or "C2SLS"; see ‘Details’.

uhet

Optional one-sided formula of environmental variables, giving σU,i=σUexp⁡(qi′δ)\sigma_{U,i} = \sigma_U \exp(q_i'\delta). Supplying it selects the 2017 model; omitting it gives the 2016 one. Variables may appear in both uhet and endogenous. Not available for "C2SLS", whose moment correction is homoskedastic; supplying it there is an error.

inefdec

TRUE (default) for a production frontier, where inefficiency is subtracted; FALSE for a cost frontier.

maxit.bobyqa, maxit.psoptim, maxit.optim

Iteration caps for the three optimizer stages. Ignored by "C2SLS", which is closed-form.

start_val

Name the starting-value vector in the returned object.

PSopt

Run the particle-swarm stage between BOBYQA and optim.

optHessian

Compute the Hessian, and with it the standard errors.

Method

Method passed to optim for the final stage. Must be "L-BFGS-B": it is the only optim method that applies the lower and upper bounds this package uses to keep scale parameters positive, and any other value is refused with an error rather than quietly optimized without them.

verbose

Report optimizer progress.

keep_objective

Logical. Retain the likelihood closure on the fitted object as $objective, so the score-based tools can difference it: vcov(type = "bhhh") (OPG/BHHH standard errors), influence_sfa, the sandwich methods, TIC and vuong. New in 1.2.1, for "IVLIML" and "IVCF". It has no effect for "C2SLS", which is two-stage least squares with a corrected intercept rather than maximum likelihood and so has no score; that case warns. For "IVCF" the reduced-form parameters are held at their OLS values, so the resulting standard errors – like the Hessian ones – condition on the first stage rather than correcting for it. FALSE by default, because a closure carries its enclosing environment.

rand.psoptim

Optional seed for the particle-swarm stage.

Details

Formula syntax

Unlike the rest of the package neither formula nor endogenous nor instruments takes a | segment. A pipe means “variance determinant” in sfm and psfm, and reusing it here for ivreg-style instrument syntax would give the same character two meanings across entry points, so a pipe is an error rather than being reinterpreted.

The model

Writing pip_i for the endogenous block and ziz_i for the instruments,

yi=β′xi+vi−ui,pi=Π′zi+ξiy_i = \beta'x_i + v_i - u_i, \qquad p_i = \Pi'z_i + \xi_i

ui=σU,i∣Ui∣,Ui∼N(0,1),(vi,ξi′)∼N(0,Ω)u_i = \sigma_{U,i}|U_i|, \quad U_i \sim N(0,1), \qquad (v_i, \xi_i') \sim N(0, \Omega)

with uiu_i independent of (vi,ξi)(v_i, \xi_i). Endogeneity is the correlation between ξ\xi and vv: the regressors are correlated with the noise, not with the inefficiency.

Conditioning vv on ξ\xi turns this into an ordinary normal–half-normal frontier in a shifted residual with a smaller noise variance,

μc,i=ΣvξΣξξ−1ξi,σc2=σV2−ΣvξΣξξ−1Σξv\mu_{c,i} = \Sigma_{v\xi}\Sigma_{\xi\xi}^{-1}\xi_i, \qquad \sigma_c^2 = \sigma_V^2 - \Sigma_{v\xi}\Sigma_{\xi\xi}^{-1}\Sigma_{\xi v}

which is what makes the likelihood closed-form and needs no simulation.

The estimators

"IVLIML"

APS (2016) Equation (13): maximum likelihood over the frontier and the reduced form jointly, so Π\Pi and Σξξ\Sigma_{\xi\xi} are estimated alongside β\beta. Efficient, and its standard errors are correct as reported. It is seeded from the "IVCF" solution rather than from t=0t = 0, which is why fitting it runs a cheap control-function fit first. t=0t = 0 is ρ=0\rho = 0 exactly – the point where the frontier and reduced-form blocks are independent, so the likelihood has no gradient in ρ\rho there – and it is also the centre of the search box, which put the corner ρ=−10/101\rho = -10/\sqrt{101} exactly as far from the start as the true optimum. Through version 1.2.0 the optimizer took that corner on a minority of samples and reported the result as an ordinary fit. Since "IVCF" is this same likelihood with Π\Pi and Σξξ\Sigma_{\xi\xi} held at least squares, its answer is a point of "IVLIML"'s own parameter space, not an approximation carried in from elsewhere.

"IVCF"

The two-step control function of Kutlu (2010), described in APS (2016) Section 4.4. Π\Pi and Σξξ\Sigma_{\xi\xi} are fixed at their reduced-form least-squares values and only the frontier block is maximized. Cheaper, still consistent, but it discards the information about the reduced form carried in the frontier likelihood; its conventional standard errors understate the true ones unless Σvξ=0\Sigma_{v\xi} = 0. Measured over 30 replications at n=1000n = 1000: with ρ=0.6\rho = 0.6 the mean reported standard error on the endogenous coefficient is 0.912 of the Monte Carlo standard deviation, while "IVLIML"'s is 1.005; with ρ=0\rho = 0 the two are identical, as the theory says they should be. Bootstrap, or use "IVLIML", when the standard errors matter.

"C2SLS"

APS (2016) Section 4.1: 2SLS, then σU\sigma_U and σV\sigma_V from the second and third moments of the residuals, then the intercept corrected by 2/π σ^U\sqrt{2/\pi}\,\hat\sigma_U. The residuals use the actual endogenous regressors, not their fitted values, which the paper flags explicitly. Moment-based rather than likelihood-based, so the fit carries no $opt and logLik returns NA with a warning – the same footing as psfm's "GTRE_SEQ1". It estimates no ρ\rho, since it never models the correlation it corrects for.

Inefficiency

jlms implements APS (2016) Equations (14)–(15), which condition on ξ\xi as well as on ε\varepsilon. Since ξ\xi is correlated with vv it carries information about uu even though uu is independent of ξ\xi, so this is a strictly better predictor than the plain Jondrow et al. (1982) formula.

What is assumed, and what is not covered

uu is assumed independent of (v,ξ)(v, \xi). The case where the reduced-form error is correlated with the inefficiency as well requires the copula likelihood of APS (2016) Section 4.5 and is not implemented; the paper notes there that two intuitive approaches to that case do not work.

Standard errors for ρ\rho

Standard errors for ρ\rho arrived in 1.2.0. They were previously reported as NA, on the ground that ρ=t/1+t′t\rho = t/\sqrt{1+t't} has a non-diagonal Jacobian and a delta-method value computed as though it were diagonal would be wrong. That much was true, but the Jacobian itself is short – J=(I−ρρ′)/sJ = (I - \rho\rho')/s with s=1+t′ts = \sqrt{1+t't} – so Var(ρ^)=J Var(t^) J′\mathrm{Var}(\hat\rho) = J\,\mathrm{Var}(\hat t)\,J' is available exactly. Checked against a numerical Jacobian in the tests, and validated end-to-end by the size of endogeneity_test, which is 0.046 at a nominal 5% over 1000 replications.

Value

An object of class "sfareg". out is a p x 3 matrix, one row per parameter, columns par, st_err and t-val. For "IVLIML" and "IVCF" the parameters are the frontier coefficients, sigma_u, sigma_v, any delta_ terms from uhet, and one rho_ per endogenous variable. For "C2SLS" they are sigma_u and sigma_v first, then the frontier coefficients, all with NA standard errors; it reports no rho_, and the fit carries no rho, vcov_rho or sigma_c. Alongside the usual components:

Pi

Reduced-form coefficient matrix: estimated jointly for "IVLIML", the least-squares value for "IVCF" and "C2SLS".

Sigma_xi

Reduced-form error covariance, on the same footing as Pi.

rho

Correlation of each reduced-form error with vv.

sigma_c

The conditional noise standard deviation σc\sigma_c, which is smaller than sigma_v whenever ρ≠0\rho \ne 0.

b_2sls

The 2SLS coefficients, which seed the likelihood and are the estimate "C2SLS" corrects.

wrong_skew

Whether the 2SLS residuals are skewed the wrong way.

jlms, efficiency

E[ui∣εi,ξi]E[u_i \mid \varepsilon_i, \xi_i] and exp(-jlms).

vcov_rho

The full q×qq \times q covariance of ρ^\hat\rho, which endogeneity_test needs for the joint Wald statistic.

References

Amsler, C., Prokhorov, A. and Schmidt, P. (2016). Endogeneity in stochastic frontier models. Journal of Econometrics, 190(2), 280–288.

Amsler, C., Prokhorov, A. and Schmidt, P. (2017). Endogenous environmental variables in stochastic frontier models. Journal of Econometrics, 199(2), 131–140.

Kutlu, L. (2010). Battese-Coelli estimator with endogenous regressors. Economics Letters, 109(2), 79–81.

See Also

endogeneity_test for the Wald test of whether the correction was needed at all; sfm for the frontier without one.

Examples

set.seed(1)
n   <- 600
w1  <- rnorm(n); w2 <- rnorm(n); x1 <- rnorm(n)
eta <- rnorm(n)
v   <- 0.5 * (0.6 * eta + sqrt(1 - 0.6^2) * rnorm(n))
x2  <- 0.9 * w1 - 0.7 * w2 + 0.5 * x1 + eta
y   <- 0.5 + 0.8 * x1 - 0.6 * x2 + v - abs(rnorm(n))
dat <- data.frame(y = y, x1 = x1, x2 = x2, w1 = w1, w2 = w2)

fit <- ivsfm(y ~ x1 + x2, endogenous = ~x2, instruments = ~ w1 + w2,
             data = dat)
fit$out

## Ignoring the endogeneity biases the coefficient on x2 toward zero.
sfm(y ~ x1 + x2, model_name = "NHN", data = dat)$out

Latent Class Stochastic Frontier Models

Description

Fits finite-mixture stochastic frontiers: models in which each firm belongs to one of JJ unobserved classes, and the likelihood mixes the classes' normal/half-normal densities weighted by the class probabilities. Three specifications:

  • "LCM": the latent class frontier of Greene (2005) and Orea and Kumbhakar (2004). Each class has its own frontier and its own σv\sigma_v and σu\sigma_u; the class probabilities are constant.

  • "LCM_Z": "LCM" with the class probabilities depending on covariates, given in the second part of the formula.

  • "LCM_CN": the contaminated normal frontier. One frontier and one σu\sigma_u; only the noise scale differs between components, which gives the noise heavier tails.

Usage

lcsfm(formula, model_name = c("LCM", "LCM_Z", "LCM_CN"), 
data, n_class = 2, maxit.bobyqa = 10000, maxit.psoptim = 1000, 
maxit.optim = 1000, REPORT = 1, trace = 0, pgtol = 0, 
start_val = FALSE, PSopt = FALSE, optHessian = TRUE, 
inefdec = TRUE, penalty_c = 0, upper = NA, Method = "L-BFGS-B", 
verbose = FALSE, keep_objective = FALSE, rand.psoptim = NULL)

Arguments

formula

A symbolic description of the frontier, y ~ x1 + x2. For "LCM_Z" a second part gives the covariates driving class membership: y ~ x1 + x2 | z1.

model_name

"LCM", "LCM_Z" or "LCM_CN"; see ‘Description’. Matching ignores case, so "lcm" and "LCM" are the same choice.

data

A data frame containing the variables named in formula.

n_class

Integer. The number of latent classes JJ, at least 2. Defaults to 2. With kk frontier coefficients and qq class-membership covariates (q=1q = 1 for the constant under "LCM"), "LCM" and "LCM_Z" estimate J(k+2)+(J−1)qJ(k + 2) + (J-1)q parameters, and "LCM_CN" estimates J+1+k+(J−1)J + 1 + k + (J-1). A large JJ needs a correspondingly large sample.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine.

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine.

maxit.optim

Maximum number of iterations for the optim optimization routine.

REPORT, trace, pgtol, upper

Accepted for compatibility with the other entry points' signatures, but not used by lcsfm(): the optimizer's own tracing is controlled by verbose, and its bounds are set internally.

start_val

FALSE (the default) to build starting values from the data (see ‘Label switching’), or a numeric vector giving them directly, in the order of the rows of out.

PSopt

Logical. Run the particle-swarm stage (pso::psoptim) between bobyqa and optim? Defaults to FALSE.

optHessian

Logical. Finish with optim() and a numerically differentiated Hessian? Standard errors and t-values require TRUE, the default.

inefdec

Logical. TRUE (the default) for a production frontier, ε=v−u\varepsilon = v - u; FALSE for a cost frontier, ε=v+u\varepsilon = v + u.

penalty_c

Tuning constant for Chen et al.'s (2001) penalty clog⁡(JJ∏jpj)c\log(J^J\prod_j p_j), added to the log-likelihood so that no class probability can be driven to 0 or 1. The default 0 leaves the ordinary likelihood untouched. A positive value is what lcsfm_homogeneity needs, and it changes what is maximised: $opt$value then holds the penalised objective, while $logLik_unpenalised and $penalty carry the two pieces separately. Available for "LCM" and "LCM_CN"; refused for "LCM_Z", whose class probabilities vary by firm, since the penalty is defined on a scalar mixing proportion.

Method

Method passed to optim for the final stage. Must be "L-BFGS-B": it is the only optim method that applies the lower and upper bounds this package uses to keep scale parameters positive, and any other value is refused with an error rather than quietly optimized without them.

verbose

Logical. Print optimization progress messages? Default is FALSE.

keep_objective

Logical. Retain the likelihood closure on the fitted object as $objective, so the score-based tools can difference it: vcov(type = "bhhh") (OPG/BHHH standard errors), influence_sfa, the sandwich methods, TIC and vuong. New in 1.2.1. The per-observation contributions are those of the mixture likelihood, log⁡∑jpjfj(yi)\log \sum_j p_j f_j(y_i), not of any one class. With penalty_c > 0 they are unpenalised – a penalty on the class probabilities is not a per-observation quantity – so the OPG is that of the unpenalised likelihood; at the default penalty_c = 0 the two coincide. FALSE by default, because a closure carries its enclosing environment.

rand.psoptim

Integer. Seed for replication of psoptim. Defaults to NULL.

Details

The latent class frontier ("LCM", "LCM_Z")

JJ distinct technologies are present in one sample, and which one a firm operates is unobserved. Each class jj has its own frontier and its own two scales,

yi=xi′βj+vij−uij,vij∼N(0,σvj2),uij∼N+(0,σuj2),y_i = x_i'\beta_j + v_{ij} - u_{ij}, \quad v_{ij} \sim N(0, \sigma_{vj}^2), \quad u_{ij} \sim N^+(0, \sigma_{uj}^2),

and class membership follows a multinomial logit,

P(j∣qi)=exp⁡(qi′δj)/∑mexp⁡(qi′δm),δJ=0,P(j \mid q_i) = \exp(q_i'\delta_j) / \sum_m \exp(q_i'\delta_m), \quad \delta_J = 0,

with class JJ the reference. Under "LCM" the qiq_i are a constant, so every firm has the same class probabilities; under "LCM_Z" they are the covariates in the second part of the formula. Every firm contributes to every class, weighted by its class probability, so the log-likelihood mixes the JJ composed densities rather than assigning firms to groups:

log⁡Li=log⁡∑jP(j∣qi)fj(εij).\log L_i = \log \sum_j P(j \mid q_i) f_j(\varepsilon_{ij}).

This is not the same as fitting separate frontiers to known subgroups: the groups are unknown, and the uncertainty about them is carried into the efficiency predictions. It is also more general than zsfm's zero-inefficiency model, which is the restricted two-class case in which one class has no inefficiency at all.

Efficiency predictions

post.prob is an n×Jn \times J matrix of posterior class probabilities. jlms_class holds the class-conditional Jondrow et al. (1982) predictions, and jlms is their posterior-weighted average. A firm's inefficiency is only defined relative to a frontier, and which frontier it faces is itself uncertain, so reporting the modal class alone would discard that. class gives the modal class, and class_prob the average prior class shares.

The contaminated normal frontier ("LCM_CN")

The same machinery, used to give the noise heavier tails rather than to separate technologies. The noise density is a scale mixture of normals, fv(v)=∑jpjϕ(v/σvj)/σvjf_v(v) = \sum_j p_j \phi(v/\sigma_{vj})/\sigma_{vj}, while the frontier, σu\sigma_u and the mixing proportions are common to all components. At J=2J = 2 this is the scale-contaminated normal, an alternative to a Student-tt noise term.

It has a closed form, which is why it is cheap. Since fε(e)=∫fv(e+u)fu(u) duf_\varepsilon(e) = \int f_v(e + u) f_u(u)\,du is linear in fvf_v, a mixture noise density passes straight through:

fε(e)=∑jpj fNHN(e;σvj,σu),f_\varepsilon(e) = \sum_j p_j\, f_{NHN}(e; \sigma_{vj}, \sigma_u),

a mixture of ordinary normal/half-normal densities sharing one σu\sigma_u. No extra integration is required, and the tests check the identity against direct numerical integration.

The parameter vector is (σv1,…,σvJ,σu,β,logit)(\sigma_{v1},\dots,\sigma_{vJ}, \sigma_u, \beta, \mathrm{logit}) rather than one block per class. There is one frontier and one σu\sigma_u, so there are no class-conditional efficiency predictions: jlms, jlms_class and class are not returned.

This is also the specification lcsfm_homogeneity's asymptotic null is stated for: one scalar parameter differing between components. On "LCM", where everything varies, the χ0:12\chi^2_{0:1} null rejected 63.5% of the time at a nominal 5% under a true null; on "LCM_CN" it rejected 8.0%. That is still mildly liberal, and the test says so, but usable.

Label switching

Class labels in a finite mixture are identified only up to permutation: relabelling the classes and permuting δ\delta gives the identical likelihood. Starting values split the OLS residuals at their JJ-quantiles and refit within each group. That makes a run reproducible on the same data, and it avoids the saddle point that identical starting components would sit at (equal posterior class probabilities give a zero score with respect to the class split). It does not pin the labels to any external ordering. Do not compare class 1 across two fits, or against a simulation's true labels, without matching the classes first.

Choosing JJ

The likelihood cannot fall when a class is added, and the usual regularity conditions for a likelihood-ratio test of JJ against J+1J+1 do not hold, since the null puts a class probability on the boundary. Use AIC()/BIC(), the separation of the fitted frontiers, and whether the extra class is economically interpretable, rather than a p-value. Whether any latent classes are needed at all – one class against the fitted JJ – is the exception: lcsfm_homogeneity tests it for "LCM" and "LCM_CN" fits.

Value

An object of class "sfareg" containing the following components:

out

A matrix with parameter estimates, standard errors, and t-values, one row per parameter. Under "LCM" and "LCM_Z" the class blocks are named <par>_class<j> (sigv_class1, sigu_class1, x1_class1, ...). Under "LCM_CN" they are sigv_class1, ..., sigv_class<J>, then sigu and the frontier coefficients unsuffixed. The class-probability coefficients are logit_<var>_class<j> for j<Jj < J.

opt

A list containing the optimization results from the final optimization procedure. With penalty_c > 0, opt$value is the penalised objective.

total_time

The total computation time for model estimation.

start_v

The final parameter vector, in the order of the rows of out.

model_name

The specification fitted: "LCM", "LCM_Z" or "LCM_CN".

formula

The formula used in the model specification.

jlms, jlms_class, class

"LCM" and "LCM_Z" only. The posterior-weighted Jondrow et al. (1982) inefficiency predictions; an n×Jn \times J matrix of the class-conditional ones; and each observation's modal class. See ‘Efficiency predictions’.

post.prob

An n×Jn \times J matrix of posterior class probabilities, whose rows sum to one.

class_prob

Average prior class probabilities, one per class.

n_class

The number of classes fitted.

penalty, logLik_unpenalised, penalty_c

The penalty term at the estimate, the log-likelihood without it, and the penalty_c used. logLik() reports logLik_unpenalised. At the default penalty_c = 0 the penalty is exactly 0.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE).

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

call

The matched call.

nobs

The number of observations used in the fit.

objective

The likelihood closure, when keep_objective = TRUE.

Note

Standard errors require optHessian = TRUE, the default.

Author(s)

Christopher F. Parmeter and David H. Bernstein

References

Greene, W. (2005) 'Reconsidering heterogeneity in panel data estimators of the stochastic frontier model', Journal of Econometrics, 126(2), pp. 269-303. doi:10.1016/j.jeconom.2004.05.003.

Orea, L. and Kumbhakar, S.C. (2004) 'Efficiency measurement using a latent class stochastic frontier model', Empirical Economics, 29(1), pp. 169-183. doi:10.1007/s00181-003-0184-2.

Caudill, S.B. (2003) 'Estimating a mixture of stochastic frontier regression models via the em algorithm: a multiproduct cost function application', Empirical Economics, 28(3), pp. 581-598. doi:10.1007/s001810200142.

Chen, H., Chen, J. and Kalbfleisch, J.D. (2001) 'A modified likelihood ratio test for homogeneity in finite mixture models', Journal of the Royal Statistical Society: Series B, 63(1), pp. 19-29.

Jondrow, J., Lovell, C.A.K., Materov, I.S. and Schmidt, P. (1982) 'On the estimation of technical inefficiency in the stochastic frontier production function model', Journal of Econometrics, 19(2-3), pp. 233-238. doi:10.1016/0304-4076(82)90004-5.

See Also

lcsfm_homogeneity, zsfm, sfm, psfm, ttsfm, sfareg-methods

Examples

library(sfa)

## Two coexisting technologies, class probabilities constant.
set.seed(42)
n   <- 400
x1  <- rnorm(n); x2 <- rnorm(n)
cls <- rbinom(n, 1, 0.4) + 1L
b   <- rbind(c(1, 0.5, 0.5), c(3, 1.0, 0.2))
sv  <- c(0.2, 0.2); su <- c(0.5, 1.0)
y   <- b[cls, 1] + b[cls, 2] * x1 + b[cls, 3] * x2 +
       rnorm(n, 0, sv[cls]) - abs(rnorm(n, 0, su[cls]))

fit <- lcsfm(y ~ x1 + x2, model_name = "LCM",
             data = data.frame(y, x1, x2), n_class = 2)
fit$class_prob        # average class shares
head(fit$post.prob)   # posterior class membership, one row per firm

Test a latent class frontier against homogeneity

Description

Tests whether a lcsfm fit's latent classes are needed at all, against the null that one class describes the data. This is the first question a latent class fit raises and the ordinary likelihood ratio test cannot answer it, because under the null the second class's parameters are unidentified, the mixing proportion sits on a boundary, and the classes are interchangeable.

Usage

lcsfm_homogeneity(object, null = c("bootstrap", "chisq01"),
                  B = 199, c = 1, level = 0.05,
                  seed = NULL, quiet = FALSE, envir = parent.frame())

Arguments

object

An "sfareg" fit from lcsfm with model_name = "LCM" or "LCM_CN" and n_class >= 2. "LCM_Z" is refused, since its class probabilities vary by firm.

null

How to obtain the null distribution. "bootstrap" (the default) is a parametric bootstrap under the fitted one-class model. "chisq01" uses the published χ0:12\chi^2_{0:1} limit and warns: see ‘Details’ for why it does not apply to "LCM" as implemented.

B

Bootstrap replications. Each is two refits, so this dominates the running time. The smallest attainable p-value is 1/(B+1)1/(B+1).

c

Tuning constant in the penalty clog⁡(JJ∏jpj)c\log(J^J\prod_j p_j). Chen et al. (2001) report their simulations were insensitive to it; Stead, Wheat and Greene use 1 and 5.

level

Size used for the reported verdict.

seed

Optional integer seed for the bootstrap. The RNG state is saved and restored, so a seeded call does not disturb the caller's stream.

quiet

Set TRUE to suppress the bootstrap progress messages.

envir

Environment in which to re-evaluate the fit's data argument.

Details

The statistic. lcsfm is refitted maximising the modified likelihood – the ordinary log-likelihood plus Chen et al.'s penalty clog⁡(JJ∏jpj)c\log(J^J\prod_j p_j), which diverges to −∞-\infty as any class probability approaches 0 or 1 and so keeps every class occupied. The statistic is twice the difference between that maximum and the maximised log-likelihood of the one-class model, which is sfm(model_name = "NHN") on the same formula, since a single latent class is the ordinary frontier. The penalty is exactly zero at equal class probabilities, so the null model is recovered without a handicap.

Why the default null is a bootstrap. Stead, Wheat and Greene report the modified statistic is asymptotically χ0:12\chi^2_{0:1}, a 50:50 mixture of a point mass at zero and χ12\chi^2_1. That result is stated for the case in which a scalar parameter differs between classes – Zhu and Zhang (2004) require θ1\theta_1 and θ2\theta_2 scalar, and the paper's own application holds every parameter except the noise scale common across classes.

lcsfm's "LCM" does not do that: it lets σv\sigma_v, σu\sigma_u and the whole frontier vector vary by class. The limit therefore does not carry over, and this was measured rather than assumed. Over 200 replications of a one-class data generating process at n=400n = 400:

nominal size 10% 5%
actual rejection under "chisq01" 82.0% 63.5%

with the sampling distribution tracking χ52\chi^2_5 – exactly the parameter-count difference – rather than χ0:12\chi^2_{0:1}, whose median is 0 against an observed 4.21.

A parametric bootstrap makes no appeal to an asymptotic distribution: it simulates from the fitted null and re-runs the whole procedure, so it is valid whatever the parameter structure. Over 60 replications of the same one-class process with B = 99:

nominal size 10% 5%
"chisq01" 82.0% 63.5%
"bootstrap" 10.0% 6.7%

both within a binomial standard error of nominal, with the p-values indistinguishable from uniform (KS against Uniform(0,1), p=0.80p = 0.80). On a sample that "chisq01" rejected at p=0.005p = 0.005, the bootstrap returns p=0.32p = 0.32, and on genuinely two-class data the statistic is 17 times the largest of 99 null draws.

Cost. The bootstrap is B+1B+1 pairs of refits. Budget accordingly, and use seed for reproducibility.

"LCM_Z" is refused, by both this function and lcsfm(penalty_c = ): it makes the class probabilities depend on covariates, and the penalty is defined on a scalar mixing proportion.

Value

An object of class c("sfa_mlrt", "htest"), so it prints like any other R test. Alongside statistic and p.value it carries null, B, boot (the bootstrap null draws), boot_ok, penalty_c, penalty_at_max, the penalised and unpenalised JJ-class log-likelihoods, logLik_1, class_prob, and reject.

References

Chen, H., Chen, J. and Kalbfleisch, J.D. (2001). A modified likelihood ratio test for homogeneity in finite mixture models. Journal of the Royal Statistical Society B, 63(1), 19–29.

Stead, A.D., Wheat, P. and Greene, W.H. (2023). On hypothesis testing in latent class and finite mixture stochastic frontier models, with application to a contaminated normal-half normal model. Journal of Productivity Analysis, 60(1), 37–48.

Zhu, H.-T. and Zhang, H. (2004). Hypothesis testing in mixture regression models. Journal of the Royal Statistical Society B, 66(1), 3–16.

See Also

lcsfm, TIC, skewness_test

Examples

set.seed(21)
n   <- 300
x1  <- rnorm(n)
cls <- rbinom(n, 1, 0.5)
d   <- data.frame(y = ifelse(cls == 1, 4, 1) + x1 +
                      rnorm(n, 0, 0.4) - abs(rnorm(n, 0, 0.6)),
                  x1 = x1)

f <- lcsfm(y ~ x1, model_name = "LCM", data = d, n_class = 2)

## Small B for the example only; use the default in real work.
lcsfm_homogeneity(f, B = 19, seed = 1, quiet = TRUE)

Marginal Effects of the Variance Determinants on Inefficiency

Description

For a model whose inefficiency scale depends on covariates (the _Z models), the marginal effects of those covariates on the mean and variance of the inefficiency term, ∂E[u]/∂zk\partial E[u]/\partial z_k and ∂Var[u]/∂zk\partial \mathrm{Var}[u]/\partial z_k.

Fit Link on σu\sigma_u Covariates reported
sfm() "NHN_Z", "NE_Z" SD (default) or z_link = "var" those in z
sfm() "NTN" with uhet/muhet SD (default) or z_link = "var" those in either design
sfm() "NTN" with scaling one factor h(z)h(z) those in scaling
psfm() "TRE_Z", "GTRE_Z" variance those in z (or zp, component = "h")

Usage

marginal_effects(object, average = FALSE, component = c("u", "h"))

Arguments

object

an object of class "sfareg" from sfm ("NHN_Z", "NE_Z", or "NTN" fitted with uhet/muhet) or psfm ("TRE_Z", "GTRE_Z"), fitted with a variance-determinant formula such as y ~ x1 + x2 | z.

component

which inefficiency component to differentiate. "u" (the default) is the only choice for every model except psfm(model_name = "GTRE_Z"), where it is the transient component; "h" selects that model's persistent component, parameterized by the third formula segment y ~ x | z | zp. The output columns are named after the component, so a table cannot be misread once separated from the call that produced it.

average

if TRUE, return only the named vector of average marginal effects. Default FALSE, which returns the full per-observation table with that vector attached as the "average" attribute.

Details

A δ\delta coefficient is not interpretable on its own: it sits in a log link for a scale parameter, so its units are neither those of uu nor those of zz, and its magnitude is not comparable across models whose link differs. The marginal effect is on the scale of uu, and is what applied work reports.

Writing s=σu(z)s = \sigma_u(z), every derivative reduces to ∂s/∂zk\partial s/\partial z_k times a constant that cancels back into the moment itself. For the half-normal and the exponential alike:

SD link:∂E[u]/∂zk=δkE[u],∂Var[u]/∂zk=2δkVar[u]\textrm{SD link:} \quad \partial E[u]/\partial z_k = \delta_k E[u], \qquad \partial \mathrm{Var}[u]/\partial z_k = 2\delta_k \mathrm{Var}[u]

variance link:∂E[u]/∂zk=(δk/2)E[u],∂Var[u]/∂zk=δkVar[u]\textrm{variance link:} \quad \partial E[u]/\partial z_k = (\delta_k/2) E[u], \qquad \partial \mathrm{Var}[u]/\partial z_k = \delta_k \mathrm{Var}[u]

The truncated normal does not reduce this way. For model_name = "NTN" the pre-truncation mean μ\mu enters E[u]E[u] alongside the scale, so E[u]E[u] is not proportional to σu\sigma_u and the one-line forms above do not apply. With a=μ/σua = \mu/\sigma_u and λ(a)=ϕ(a)/Φ(a)\lambda(a) = \phi(a)/\Phi(a),

E[u]=μ+σuλ(a),Var[u]=σu2(1−λ(a)(λ(a)+a))E[u] = \mu + \sigma_u \lambda(a), \qquad \mathrm{Var}[u] = \sigma_u^2 \left(1 - \lambda(a)(\lambda(a) + a)\right)

and, since λ′(a)=−λ(a)(λ(a)+a)\lambda'(a) = -\lambda(a)(\lambda(a) + a),

∂E[u]/∂μ=1−λ(λ+a),∂E[u]/∂σu=λ+aλ(λ+a).\partial E[u]/\partial \mu = 1 - \lambda(\lambda + a), \qquad \partial E[u]/\partial \sigma_u = \lambda + a\lambda(\lambda + a).

The reported effect is the chain rule through both designs, since μ\mu and σu\sigma_u may depend on different covariates: ∂μ/∂zk=δkμ\partial \mu/\partial z_k = \delta^{\mu}_k (an identity link) and ∂σu/∂zk\partial \sigma_u/\partial z_k as above. A covariate appearing in one design but not the other contributes only through that one, and every non-constant covariate of either design gets a column, so muhet = ~ z alone, with a constant σu\sigma_u, still reports zz. Note that δμ\delta^{\mu} is therefore not a semi-elasticity, unlike the scale coefficients.

The scaling property is exact and needs no family branch. Under sfm(scaling = ~ z) — implemented for model_name = "NTN" only, since for a half-normal h∣N(0,σ2)∣h|N(0,\sigma^2)| is ∣N(0,(hσ)2)∣|N(0,(h\sigma)^2)| and uhet already fits it — one factor h(z)=exp⁡(z′δs)h(z) = \exp(z'\delta_s) multiplies both the scale and the pre-truncation mean. So a=μ/σua = \mu/\sigma_u is constant in zz, E[u]=h E[u∗]E[u] = h\,E[u^*], and

∂E[u]/∂zk=δs,kE[u],∂Var[u]/∂zk=2δs,kVar[u]\partial E[u]/\partial z_k = \delta_{s,k} E[u], \qquad \partial \mathrm{Var}[u]/\partial z_k = 2\delta_{s,k} \mathrm{Var}[u]

whatever the inefficiency family is. Here δs,k\delta_{s,k} is the semi-elasticity ∂log⁡E[u]/∂zk\partial \log E[u]/\partial z_k, so dE\_u.dz / E\_u is constant across observations by construction.

The factor of two matters. sfm()'s "NHN_Z" and "NE_Z" place the linear predictor on the standard deviation, σu=exp⁡(z′δ)\sigma_u = \exp(z'\delta), while psfm()'s "TRE_Z" and "GTRE_Z" place it on the variance, σu=exp⁡(z′δ)\sigma_u = \sqrt{\exp(z'\delta)}. Reading a δ\delta from one family with the other's convention in mind is wrong by exactly this factor. Reporting the marginal effect instead of the coefficient removes the trap, since the effect is on the scale of uu either way.

The link is read from the fit rather than assumed, so a sfm model fitted with z_link = "var" and a psfm model are handled the same way and their effects are directly comparable.

psfm(model_name = "GTRE_Z") separates persistent inefficiency hih_i from transient uitu_{it} and gives each its own determinant block, so both are available: component = "u" for the transient effects and component = "h" for the persistent ones. Each block is located by NAME rather than by position, because the σh\sigma_h coefficients are the trailing ones and a positional rule would report them under a σu\sigma_u label.

Effects are per observation, because σu\sigma_u varies with zz. Constant columns of the zz design are dropped: the derivative with respect to an intercept is not a marginal effect.

Standard errors are deliberately not reported. The effect is a nonlinear function of both δ\delta and zz, so its sampling distribution requires the delta method through the full covariance matrix or a bootstrap; a number printed without one would be read as inference.

Value

With average = FALSE, a data frame with one row per observation: sigma_u, E_u, Var_u, then one dE_u.d<z> and one dVar_u.d<z> column per non-constant variance determinant. With component = "h" the same columns are named sigma_h, E_h, Var_h, dE_h.d<z> and dVar_h.d<z>. Attributes "average" (the average marginal effects), "link", "family" and "component" record the convention used.

With average = TRUE, just the named vector of average marginal effects.

See Also

sfm, efficiency_ci

Examples

dat <- data_gen_cs(N = 300, rand = 1, sig_u = 1, sig_v = 0.3,
                   cons = 0.5, beta1 = 0.5, beta2 = 0.5, a = 5, mu = 0.5)
fit <- sfm(y_pcs_z ~ x1 + x2 | z, data = dat, model_name = "NHN_Z")

me <- marginal_effects(fit)
head(me)

## The conventional summary.
marginal_effects(fit, average = TRUE)

Model-Implied Mean Efficiency

Description

The mean efficiency implied by the fitted distribution of inefficiency, E[exp⁡(−U)]E[\exp(-U)], and the mean efficiency among the most efficient pp of that distribution.

Usage

meanefficiency(object,
               p = c(0.01, 0.05, 0.10, 0.25, 0.50, 0.75, 0.90, 0.95, 0.99),
               use_closed_form = TRUE)

Arguments

object

An "sfareg" fit.

p

Proportions in (0,1](0, 1] at which to report supra-percentile mean efficiency. p = 1 is the whole distribution and returns the unconditional mean.

use_closed_form

Use the closed-form expression where one exists. Setting it to FALSE forces numerical integration of the same density, which is how the closed forms are checked.

Details

A different quantity from the reported average. print and summary report the average of the per-observation conditional predictions E[exp⁡(−ui)∣εi]E[\exp(-u_i) \mid \varepsilon_i]. That is a property of the sample in hand. E[exp⁡(−U)]E[\exp(-U)] is a property of the fitted model, and it is the number that can be compared across studies.

Closed forms. Available for the half-normal (2eσU2/2Φ(−σU)2e^{\sigma_U^2/2}\Phi(-\sigma_U)), exponential (1/(1+σU)1/(1+\sigma_U)), truncated normal, Rayleigh, gamma ((1+θ)−k(1+\theta)^{-k}), uniform ((1−e−θ)/θ(1-e^{-\theta})/\theta) and generalized exponential (2/((σU+1)(σU+2))2/((\sigma_U+1)(\sigma_U+2))). The lognormal, Weibull and Nakagami have none in elementary terms and are integrated numerically; method in the returned list says which route was taken.

Every closed form is checked against numerical integration of the same density in the package's tests, and the density is separately checked against its own quantile function by simulation. The first catches an algebraic slip; the second catches a parameter being read off the fit incorrectly, which would otherwise make both routes agree and both be wrong.

Supra-percentile means are always integrated. Efficiency is decreasing in uu, so the most efficient pp are the smallest pp of uu: the reported value is p−1∫0Q(p)e−uf(u) dup^{-1}\int_0^{Q(p)} e^{-u}f(u)\,du. It necessarily exceeds the unconditional mean and falls as pp rises.

Value

A list with model, distribution, the parameters used, mean_efficiency, the method used to obtain it, and supra, a data frame of p against mean efficiency.

See Also

efficiency for the per-observation predictors.

Examples

d <- data_gen_cs(N = 400, rand = 1, sig_u = 1, sig_v = 0.3, cons = 0.5,
                 beta1 = 0.5, beta2 = 0.5, a = 1, mu = 0.5)
fit <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d)

r <- meanefficiency(fit)
r$mean_efficiency
r$supra

What skewness and excess kurtosis can a pair of distributions produce?

Description

Reports, for each pair of noise and inefficiency distributions, the set of skewness and excess kurtosis values the composed error ε=v±u\varepsilon = v \pm u can attain at all, and says which pairs the residual moments rule out. This is the Type II failure check of Papadopoulos and Parmeter (2021): a pair whose attainable set does not contain the sample moments is refuted without any test statistic. It needs no frontier fit.

Usage

moment_range(object = NULL,
             noise = c("normal", "logistic", "laplace", "uniform", "t"),
             inefficiency = c("halfnormal", "exponential", "genexponential",
                              "truncatednormal", "gamma"),
             inefdec = NULL, df = 8)

Arguments

object

An "sfareg" fit, whose OLS residuals are used, or a numeric vector of residuals. Omit it to get the attainable ranges alone, as a reference table.

noise

Assumed distributions of vv; all five by default. Only the excess kurtosis of vv enters.

inefficiency

Assumed distributions of uu; all five by default.

inefdec

Orientation. Taken from the fit when there is one, otherwise production. A cost frontier puts the skewness on the positive side.

df

Degrees of freedom for noise = "t", whose excess kurtosis is 6/(ν−4)6/(\nu-4). Must exceed 4.

Details

The two equations. With R=SNR2/(1+SNR2)R = \mathrm{SNR}^2/(1+\mathrm{SNR}^2) the share of composed-error variance contributed by uu, Papadopoulos and Parmeter (2021, eqs. 5 and 10) give

∣γ1(ε)∣=γ1(u) R3/2,γ2(ε)=γ2(v)(1−R)2+γ2(u)R2.|\gamma_1(\varepsilon)| = \gamma_1(u)\,R^{3/2}, \qquad \gamma_2(\varepsilon) = \gamma_2(v)(1-R)^2 + \gamma_2(u)R^2.

Because RR lies in (0,1)(0,1), the composed skewness can never exceed the skewness of uu itself, whatever the noise: the Normal-Half-Normal pair is confined to γ1(ε)∈(−0.995,0)\gamma_1(\varepsilon) \in (-0.995, 0) and the Normal-Exponential pair to (−2,0)(-2, 0). The kurtosis equation is quadratic in RR, so its range is not simply the interval between the two components' values – with leptokurtic noise it dips below both before rising, reaching γ2(v)γ2(u)/(γ2(v)+γ2(u))\gamma_2(v)\gamma_2(u)/(\gamma_2(v)+\gamma_2(u)) at R=γ2(v)/(γ2(v)+γ2(u))R = \gamma_2(v)/(\gamma_2(v)+\gamma_2(u)).

Which uu can fail. Half-normal, exponential and generalized exponential have fixed skewness and kurtosis, so their attainable sets are the narrowest. The truncated normal is indexed by μ/σu\mu/\sigma_u and sweeps γ1(u)∈(0,2)\gamma_1(u) \in (0,2) and γ2(u)∈(−0.243,6)\gamma_2(u) \in (-0.243, 6) – it is the only one here whose composed error can be platykurtic under normal noise. The gamma has γ1(u)=2/k\gamma_1(u) = 2/\sqrt{k} and γ2(u)=6/k\gamma_2(u) = 6/k, unbounded as k→0k \to 0, so a gamma inefficiency term can never suffer a Type II failure; the price, as PP2021 note, is that k<1k < 1 makes the density log-convex.

This is a diagnostic, not a test. PP2021 section 2.3 shows the range check over-rejects, and the package's own replication of their Tables 1-5 confirms it: with correctly specified Normal-Half-Normal data at n=200n = 200 and SNR=3\mathrm{SNR} = 3, the sample excess kurtosis falls outside its theoretical range 26% of the time, and the sample skewness 16% of the time. Use spec_test for the formal statistic, which is built on the same two equations but studentised and bootstrapped.

Relation to the wrong-skew problem. A Type I failure – residual skewness of the wrong sign, Waldman (1982) – is a property of the data and is reported for every row at once. Type II failures are pair-specific: they are what tells you that the residuals are too skewed, or too heavy-tailed, for the particular uu you assumed. Wrong skew is handled by skewness_test and, as an estimator, by esfm.

Value

A data frame of class "sfa_moment_range", one row per pair, with the attainable interval for the composed skewness (g1_lo, g1_hi) and excess kurtosis (g2_lo, g2_hi), logical skew_ok and kurt_ok, and a verdict of "ok", "type I (wrong skew)" or "type II (...)". The sample moments, sample size and orientation are attributes.

References

Papadopoulos, A. and Parmeter, C. F. (2021). Type II failure and specification testing in the Stochastic Frontier Model. European Journal of Operational Research 293(3), 990-1001.

Waldman, D. M. (1982). A stationary point for the stochastic frontier likelihood. Journal of Econometrics 18(2), 275-279.

See Also

spec_test for the formal test built on the same two equations, skewness_test for the Type I failure, esfm for an estimator that tolerates it.

Examples

## The reference table on its own: no data needed.
moment_range(noise = "normal")

set.seed(4)
d <- data_gen_cs(N = 400, rand = 4, sig_u = 1, sig_v = 0.5, cons = 1,
                 beta1 = 0.5, beta2 = 0.5, a = 1, mu = 0.5)

## Exponential inefficiency, read through a normal-half-normal lens: the
## residuals are more skewed than a half-normal can make them.
fit <- sfm(y_pcs_e ~ x1 + x2, data = d, model_name = "NHN")
moment_range(fit, noise = "normal")

Nonparametric Stochastic Frontier Models

Description

Fits a stochastic frontier y=m(x)+v−uy = m(x) + v - u whose frontier m(x)m(x) is estimated by kernel methods rather than assumed linear. Five methods are available:

  • "FLW" (the default): kernel regression, then constant scale parameters from the residuals. Fast and steady.

  • "SVKZ": local moments, so σu(x)\sigma_u(x) and σv(x)\sigma_v(x) vary with the covariates. No optimizer.

  • "PSZ": local maximum likelihood, with varying σu(x)\sigma_u(x) and σv(x)\sigma_v(x). Slow.

  • "MY": iterative local likelihood, with constant scale parameters. Slow, and the most accurate in the comparison under Details.

  • "SZ": a DEA step that makes an existing smooth frontier monotone, and under rts = "vrs" or "crs" also concave.

Only "FLW" offers inefficiency distributions other than the half-normal. The result is an "npsfareg" object, with no coefficient vector or standard errors.

Usage

npsfm(formula, data, method = c("FLW", "SVKZ", "PSZ", "KPST", "MY", "SZ"),
      dist = c("hn", "exp", "gamma", "unif"),
      regtype = c("lc", "ll"), bw.sel = c("cv.ls", "cv.aic"),
      bw = NULL, cost = FALSE, eff = TRUE,
      maxit = 5000, tol = 1e-3, iter = 25,
      rts = c("vrs", "crs", "drs", "irs"),
      prior.fit = NULL, log.form = TRUE, verbose = FALSE)

Arguments

formula

A single-part symbolic description of the frontier, y ~ x1 + x2. Unlike the package's parametric entry points npsfm() takes no | z segment: heteroskedasticity is handled nonparametrically through the covariates themselves under method = "SVKZ" and "PSZ", and a pipe is an error rather than something silently ignored.

data

A data frame containing the variables named in formula.

method

Which estimator to use.

"FLW"

Fan, Li and Weersink (1996). Estimates E[y∣x]E[y|x] by kernel regression, then recovers the scale parameters from the residuals – by maximizing their concentrated likelihood in λ\lambda when dist = "hn", and by inverting central moments otherwise. σu\sigma_u and σv\sigma_v are constants; only the frontier is smooth.

"SVKZ"

Simar, Van Keilegom and Zelenyuk (2017). Runs three local-linear regressions – yy on xx, then the squared and cubed residuals on xx – and inverts the local moments, so σu(x)\sigma_u(x) and σv(x)\sigma_v(x) both vary with the covariates. No optimizer runs. Normal-half-normal only.

"PSZ" (or "KPST")

The local maximum likelihood estimator of Kumbhakar, Park, Simar and Tsionas (2007), in the form of Park, Simar and Zelenyuk (2015); hence the two names. Local maximum likelihood: at every evaluation point the frontier and both log variance components are given local-linear expansions and the kernel-weighted normal-half-normal likelihood is maximized in those 3(k+1)3(k+1) parameters. One optimization per observation, so it is much slower than the two above. σu(x)\sigma_u(x) and σv(x)\sigma_v(x) vary with the covariates.

"MY"

Martins-Filho and Yao (2015). Iterative local likelihood: alternates a local-linear fit of the frontier at every evaluation point, holding (λ,σ)(\lambda,\sigma) fixed, with a global update of (λ,σ)(\lambda,\sigma) from the resulting composed residuals, until the scale parameters stop moving. Cost is (observations ×\times iterations) optimizations. σu\sigma_u and σv\sigma_v are constants; only the frontier is local.

"SZ"

Simar and Zelenyuk (2011). Not an estimator in its own right: it takes an already-estimated smooth frontier and passes its fitted values through an output-oriented DEA, imposing the monotonicity and (under rts = "vrs"/"crs") convexity that a kernel fit does not guarantee. Supply the prior fit through prior.fit, or leave it NULL to fit "SVKZ" first. The DEA step is solved by the DEA package, which must be installed; it is implemented for production frontiers only.

Matching ignores case, so "flw" and "FLW" are the same choice.

dist

Distribution of the one-sided inefficiency term, for method = "FLW" only: "hn" (half-normal, the default), "exp" (exponential), "gamma", or "unif" (uniform on [0,b][0,b]). Every other method is derived for the half-normal case and errors for anything else.

regtype

Kernel regression type passed to np::npregbw: "lc" (local constant, the default) or "ll" (local linear). Applies to "FLW"; "SVKZ" is local linear throughout by construction.

bw.sel

Bandwidth selection method: "cv.ls" (least-squares cross-validation, the default) or "cv.aic". Ignored when bw is supplied.

bw

Optional numeric vector of bandwidths, one per covariate. When supplied, cross-validation is skipped and these are used directly – useful for undersmoothing, for sensitivity analysis, or simply to avoid paying for bandwidth selection repeatedly in a simulation.

cost

Logical. FALSE (the default) fits a production frontier, in which inefficiency is subtracted; TRUE fits a cost frontier.

eff

Logical. Compute observation-level inefficiency predictions (u_hat, exp_u_hat). Available for dist = "hn" and dist = "exp"; the gamma and uniform cases return moment estimates only. Defaults to TRUE.

maxit

Maximum function evaluations for each local optimization under "PSZ" (passed to minqa::bobyqa) and maximum iterations for each local fit under "MY" (passed to optim). Ignored by the other methods. Defaults to 5000. Raise it if convergence reports many non-zero codes.

tol

Convergence tolerance for "MY": the iteration stops when the squared change in (λ,σ)(\lambda,\sigma) falls below this. Defaults to 1e-3.

iter

Maximum number of outer iterations for "MY". Defaults to 25.

rts

Returns-to-scale assumption for the DEA step under "SZ": "vrs" (the default), "crs", "drs" or "irs". These restrict ∑jλj\sum_j \lambda_j to be unrestricted, =1= 1, ≤1\le 1 and ≥1\ge 1 respectively.

prior.fit

For "SZ": a numeric vector of already-estimated frontier values, one per observation, to be monotonized. If NULL (the default) an "SVKZ" fit is computed first and used.

log.form

For "SZ": whether the data are in logs, as is conventional in this literature. When TRUE (the default) the frontier and covariates are exponentiated before the DEA step and the result is returned to the log scale.

verbose

Logical. Report progress through the per-observation loops of "PSZ" and the outer iterations of "MY". Defaults to FALSE.

Details

All five methods keep the composed-error structure y=m(x)+v−uy = m(x) + v - u of sfm and relax only the linear frontier.

Mean shift versus local likelihood

A least-squares kernel regression of yy on xx estimates E[y∣x]=m(x)−E[u]E[y|x] = m(x) - E[u], not m(x)m(x). "FLW" and "SVKZ" start from that regression and shift it up by an estimate of E[u]E[u]. For "FLW" the shift is one number, because σu\sigma_u is constant, and the frontier gradients are those of the conditional mean. For "SVKZ", σu(x)\sigma_u(x) comes from the local third moment, so the shift varies across observations and the gradient picks up a chain-rule term.

"PSZ" and "MY" instead maximize the composed-error likelihood locally. The local intercept is then m(x)m(x) itself, and no shift is applied. The cost is one numerical optimization per observation (for "MY", per observation per iteration), so expect them to be one to two orders of magnitude slower than "FLW". Both start from an "FLW" fit.

Which to use

On a simulated nonlinear frontier with σu=0.6\sigma_u = 0.6, σv=0.25\sigma_v = 0.25 (3 replications, n=300n = 300), mean absolute frontier error was 0.070 for "MY", 0.079 for "FLW", and 0.116 for both "SVKZ" and "PSZ". "FLW" is the steadiest for its cost, and "MY" the most accurate if the run time is acceptable. "SVKZ" and "PSZ" are for the case where σu\sigma_u really varies with xx, which this design does not test.

Wrong skew

σu\sigma_u is identified by the residuals being negatively skewed. "FLW" inverts a single sample moment and either succeeds or warns. "SVKZ" inverts a local third moment, which is much noisier. Wherever its sign is wrong the implied σu(x)3\sigma_u(x)^3 is negative, and, following the paper, σu(x)\sigma_u(x) is floored at 0 there and that point's contribution to the frontier gradient is set to zero. wrong.skew flags these observations. If many are flagged, the local third moment is too noisy to inform the fit, and "FLW" is the safer choice at moderate sample sizes.

Dependencies and run time

Kernel regression and bandwidth selection come from the np package, and "SZ"'s DEA step from the DEA package. Both are in Suggests, and npsfm() stops with an install instruction if the one it needs is missing. Cross-validated bandwidth selection dominates the run time and scales quadratically in the sample size; supply bw to skip it.

Value

An object of class "npsfareg". This is deliberately not an "sfareg" object: there is no parameter vector with standard errors, so coef(), vcov() and logLik() would have nothing meaningful to return. fitted(), residuals(), nobs(), print() and summary() are provided. A component that a method does not produce is absent from the list, not NULL. Components:

frontier

The estimated frontier m^(x)\hat{m}(x). For "FLW" and "SVKZ" this is the kernel fit shifted up by the estimated E[u]E[u]; for "PSZ" and "MY" it is the local intercept; for "SZ" it is the DEA-monotonized prior fit.

frontier.grad

Matrix of estimated frontier gradients, one row per observation and one column per covariate. Under "SZ" these are the "SVKZ" gradients, and absent when prior.fit is supplied.

conditional.mean

The implied E[y∣x]E[y|x]: frontier minus mean.correction (plus, for a cost frontier). Under "SZ" it is that of the prior fit, before monotonization, and the prior fit itself when prior.fit is supplied.

residuals

Composed residuals measured against the corrected frontier, y−m^(x)y - \hat{m}(x). These are negative up to noise, unlike the centered residuals of the underlying kernel regression.

mean.correction

The estimated E[u]E[u]. A scalar under "FLW" and "MY"; a vector, one value per observation, under "SVKZ" and "PSZ". "PSZ" and "MY" report it but do not apply it to the frontier (see Details).

sigma.u, sigma.v

Estimated scale parameters. Scalars under "FLW" and "MY"; vectors of σu(xi)\sigma_u(x_i), σv(xi)\sigma_v(x_i) under "SVKZ" and "PSZ". "SZ" reports those of its "SVKZ" prior fit, or NA when prior.fit is supplied, in which case u_hat and exp_u_hat are not computed.

lambda, sigma

Returned by "FLW" with dist = "hn", and by "MY": λ=σu/σv\lambda = \sigma_u/\sigma_v and σ=σu2+σv2\sigma = \sqrt{\sigma_u^2+\sigma_v^2}.

theta

Returned by "FLW" with dist = "exp" or "gamma": the rate parameter of the one-sided term.

b

Returned by "FLW" with dist = "unif": the estimated upper bound of the uniform.

sigma.u.grad, wrong.skew

Returned by "SVKZ": the gradient of σu(x)\sigma_u(x), and a logical vector flagging observations whose local third moment had the wrong sign. "PSZ" also returns sigma.u.grad and sigma.v.grad, the local-linear slopes of the two log variance functions.

convergence

Returned by "PSZ": the minqa::bobyqa status code from each observation's local optimization, 0 for success. A large share of non-zero codes means maxit is too low and the local fits should not be trusted.

iterations, converged, tol.reached

Returned by "MY": how many outer iterations ran, whether the tolerance was met, and the final squared change in (λ,σ)(\lambda,\sigma).

prior.fit, dea.efficiency, rts

Returned by "SZ": the smooth frontier that was monotonized, the DEA efficiency scores, and the returns-to-scale assumption used.

u_hat, exp_u_hat

Jondrow et al. (1982) inefficiency predictions E[u∣ε]E[u|\varepsilon] and Battese-Coelli (1988) efficiency predictions E[exp⁡(−u)∣ε]E[\exp(-u)|\varepsilon]. Returned when eff = TRUE and dist is "hn" or "exp", and only when every σv\sigma_v is positive and finite. Under "SVKZ" (and "SZ" built on it) σv(x)\sigma_v(x) is floored at zero wherever the local variance estimate is negative, and one such point is enough for both to be omitted for every observation.

bws

The bandwidth object(s) used: one for "FLW", and for "PSZ" and "MY" that of the "FLW" fit they start from; a list of three (r1, r2, r3) for "SVKZ" and for "SZ" without prior.fit.

method, dist, formula, call, cost, regtype, bw.sel, nobs, total_time, data

Settings and bookkeeping.

Note

Requires the np package (and DEA for "SZ"), both in Suggests.

Author(s)

Christopher F. Parmeter and David H. Bernstein

References

Fan, Y., Li, Q. and Weersink, A. (1996) 'Semiparametric estimation of stochastic production frontier models', Journal of Business & Economic Statistics, 14(4), pp. 460-468.

Simar, L., Van Keilegom, I. and Zelenyuk, V. (2017) 'Nonparametric least squares methods for stochastic frontier models', Journal of Productivity Analysis, 47(3), pp. 189-204.

Kumbhakar, S.C., Park, B.U., Simar, L. and Tsionas, E.G. (2007) 'Nonparametric stochastic frontiers: a local maximum likelihood approach', Journal of Econometrics, 137(1), pp. 1-27.

Park, B.U., Simar, L. and Zelenyuk, V. (2015) 'Categorical data in local maximum likelihood: theory and applications to productivity analysis', Journal of Productivity Analysis, 43(2), pp. 199-214.

Martins-Filho, C. and Yao, F. (2015) 'Semiparametric stochastic frontier estimation via profile likelihood', Econometric Reviews, 34(4), pp. 413-451.

Simar, L. and Zelenyuk, V. (2011) 'Stochastic FDH/DEA estimators for frontier analysis', Journal of Productivity Analysis, 36(1), pp. 1-20.

Jondrow, J., Lovell, C.A.K., Materov, I.S. and Schmidt, P. (1982) 'On the estimation of technical inefficiency in the stochastic frontier production function model', Journal of Econometrics, 19(2-3), pp. 233-238.

Battese, G.E. and Coelli, T.J. (1988) 'Prediction of firm-level technical efficiencies with a generalized frontier production function and panel data', Journal of Econometrics, 38(3), pp. 387-399.

See Also

sfm for parametric cross-sectional frontiers, psfm for panel models, and data_gen_cs for simulating data with known true parameters.

Examples

if (requireNamespace("np", quietly = TRUE)) {
  set.seed(42)
  n  <- 150
  x1 <- runif(n, 1, 4)
  x2 <- runif(n, 1, 4)
  m  <- 1 + 0.6 * log(x1) + 0.4 * sqrt(x2)   # nonlinear frontier
  d  <- data.frame(y = m + rnorm(n, 0, 0.25) - abs(rnorm(n, 0, 0.6)),
                   x1 = x1, x2 = x2)

  ## Fan, Li and Weersink, normal-half-normal
  f <- npsfm(y ~ x1 + x2, data = d, method = "FLW", dist = "hn")
  f
  head(fitted(f))
  head(f$exp_u_hat)

  ## Simar, Van Keilegom and Zelenyuk: sigma_u and sigma_v vary with x
  g <- npsfm(y ~ x1 + x2, data = d, method = "SVKZ")
  summary(g$sigma.u)
}

Costs of U.S. commercial banks, 1989

Description

The dataset is a cross-section of U.S. commercial banks for 1989, extracted from the panel dataset used by Kumbhakar, Parmeter and Tsionas (2013) and based on the Federal Reserve Bank of Chicago's Reports of Condition and Income. It contains detailed cost data with inputs and outputs defined under the intermediation approach, and input prices constructed as expense-quantity ratios.

Usage

data("panel89")

Format

A data frame with 4,985 observations on the following 11 variables.

y

a numeric vector containing the natural logarithm of total cost*

q1

a numeric vector containing the natural logarithm of installment loans

q2

a numeric vector containing the natural logarithm of real estate loans

q3

a numeric vector containing the natural logarithm of business loans

q4

a numeric vector containing the natural logarithm of federal funds sold and securities purchased

q5

a numeric vector containing the natural logarithm of other assets

w1

a numeric vector containing the natural logarithm of the price of labour*

w2

a numeric vector containing the natural logarithm of the price of capital*

w3

a numeric vector containing the natural logarithm of the price of purchased funds*

w4

a numeric vector containing the natural logarithm of the price of interest-bearing deposits in total transaction accounts*

z

a numeric vector containing the natural logarithm of total assets

Details

*The cost and input price variables are normalised by that of a fifth input: the price of interest-bearing deposits in total non-transaction accounts. Total cost is defined as the sum of total expenses for each input. Input prices are derived by dividing the total expense for each input by the corresponding input quantity.

Source

Kumbhakar, S.C., Parmeter, C.F. and Tsionas, E.G. (2013) 'A zero inefficiency stochastic frontier model', Journal of Econometrics, 172(1), pp. 66-76. doi:10.1016/j.jeconom.2012.08.021.

References

Kumbhakar, S.C. and Tsionas, E.G. (2005) 'Measuring technical and allocative inefficiency in the translog cost system: a Bayesian approach', Journal of Econometrics, 126(2), pp. 355-384. doi:10.1016/j.jeconom.2004.05.006.

Examples

data(panel89)
  plot(panel89)

The distribution of the composed error

Description

Density and distribution function of the stochastic frontier composed error ε=v−u\varepsilon = v - u (production) or v+uv + u (cost), with v∼N(0,σv2)v \sim N(0, \sigma_v^2) and u∼N+(0,σu2)u \sim N^+(0, \sigma_u^2) independent. The density is a skew-normal and closed form; the distribution function is not, and is obtained here by quadrature that stays accurate far into the tails.

Usage

dcomposed(x, sigma_u, sigma_v, inefdec = TRUE, log = FALSE)

pcomposed(q, sigma_u, sigma_v, inefdec = TRUE, lower.tail = TRUE,
          log.p = FALSE, method = c("quadrature", "simulate"),
          n_nodes = 128, R = 1e6, seed = NULL)

Arguments

x, q

Numeric vector of quantiles.

sigma_u

Standard deviation of the pre-truncation normal behind uu. Zero is allowed and is the no-inefficiency boundary, where the composed error is just the noise.

sigma_v

Standard deviation of the noise, strictly positive.

inefdec

TRUE (the default) for a production frontier, ε=v−u\varepsilon = v - u; FALSE for a cost frontier, ε=v+u\varepsilon = v + u. The two are mirror images.

log, log.p

Return the log of the density or probability. Use log.p = TRUE whenever the result may be extremely small.

lower.tail

If TRUE (the default) probabilities are P(ε≤q)P(\varepsilon \le q), otherwise P(ε>q)P(\varepsilon > q). The upper tail is computed directly rather than as 1−F1 - F; see ‘Details’.

method

"quadrature" (the default) is deterministic Gauss-Legendre. "simulate" is the estimator of Amsler, Schmidt and Tsay (2019), kept for comparison.

n_nodes

Quadrature nodes, at least 16.

R, seed

Draws and optional seed for method = "simulate". The seed is restored afterwards, so the caller's random stream is untouched.

Details

Why this exists. Every likelihood in sfa evaluates the composed error's density and none needs its distribution function, which is why there was not one. Copula models need it: a Gaussian copula on the composed error contains Φ−1(F(ε))\Phi^{-1}(F(\varepsilon)), so an FF that saturates at 0 or 1 does not merely lose precision, it returns an infinity and takes the copula density with it.

How it is computed. Conditioning on uu leaves a normal distribution function in closed form, so

F(q)=Eu[Φ((q+su)/σv)],F(q) = E_u\left[\Phi\left((q + su)/\sigma_v\right)\right],

with s=+1s = +1 for a production frontier and −1-1 for a cost frontier. The noise is therefore never drawn or integrated, and the tails inherit the accuracy of pnorm, which is reliable twenty standard deviations out. The expectation over uu is taken by Gauss-Legendre quadrature on u=σut/(1−t)u = \sigma_u t/(1-t), and accumulated by log-sum-exp so that terms which would underflow individually still contribute.

The upper tail. Requesting lower.tail = FALSE negates the argument of Φ\Phi rather than forming 1−F1 - F. At q=30q = 30 with σu=σv=1\sigma_u = \sigma_v = 1 the complement is exactly 1 in double precision while the direct computation returns a finite log-probability below −102-10^{2}.

Accuracy. There is no closed form except at λ=σu/σv\lambda = \sigma_u/\sigma_v equal to 0 (the normal) or 1, where Amsler, Schmidt and Tsay show P(Q)=Φ(Q/(2σu))2P(Q) = \Phi(Q/(\sqrt{2}\sigma_u))^2 for the cost case. Against that standard the default quadrature is accurate to about 10−1310^{-13} relative at probabilities as small as 10−11510^{-115}. It is deliberately used everywhere rather than special-cased at λ=1\lambda = 1: the production form of that identity is a complement, which would reintroduce the saturation the function exists to avoid.

Value

A numeric vector the length of x or q.

References

Amsler, C., Schmidt, P. and Tsay, W.-J. (2019). Evaluating the cdf of the distribution of the stochastic frontier composed error. Journal of Productivity Analysis, 52(1), 29–35.

See Also

copsfm, sfm

Examples

## Density integrates to one.
integrate(function(x) dcomposed(x, 1, 0.5), -Inf, Inf)$value

## Distribution function, and a tail that does not saturate.
pcomposed(c(-2, 0, 2), sigma_u = 1, sigma_v = 0.5)
pcomposed(30, 1, 1, lower.tail = FALSE, log.p = TRUE)

## Checked against the one exact standard there is (cost frontier).
su <- sv <- sqrt(0.5)
c(exact = pnorm(-6 / (sqrt(2) * su))^2,
  computed = exp(pcomposed(-6, su, sv, inefdec = FALSE, log.p = TRUE)))

Composed-error distribution function for any cross-sectional model

Description

P(ε≤q)P(\varepsilon \le q) for the composed error of any cross-sectional model, where pcomposed covers only the half-normal case.

Usage

pcomposed_model(q, model_name, par, inefdec = TRUE, lower.tail = TRUE,
                log.p = FALSE, n_nodes = 128)

composed_cdf(object, q = NULL, data = NULL, ...)

Arguments

q

Numeric vector of quantiles. For composed_cdf, defaults to the fit's own composed-error residuals.

model_name

One of "NHN", "NTN", "NE", "NR", "NU", "NGE", "NLN", "NW", "NG", "NNAK", "TSL", "THT", "tHN".

par

A named parameter vector, as in fit$out[, "par"].

inefdec

TRUE for a production frontier, FALSE for cost.

lower.tail, log.p

As elsewhere in R.

n_nodes

Quadrature nodes over the inefficiency term.

object

An "sfareg" fit from sfm.

data

The data the model was fitted to. sfm() does not retain it, so it is needed whenever q is not supplied.

...

Passed to pcomposed_model.

Details

The probability is computed as an expectation over the inefficiency term,

F(q)=Eu[Fv((q+su)/σv)],F(q) = E_u\left[F_v\left((q + su)/\sigma_v\right)\right],

with s=+1s = +1 for production and −1-1 for cost. Conditioning on uu leaves the noise CDF in closed form, so the noise is never integrated and the tails inherit pnorm()'s (or pt()'s) own accuracy.

Three parameterizations do not read the way their names suggest, and are taken from each likelihood in sfm() rather than from the label: "THT" lists sigu before sigv, the only model that does; "NG"'s sigu is the gamma scale and its mu the shape; "NNAK"'s sigu is the Nakagami spread, so Ω=sigu2\Omega = \code{sigu}^2; and "NLN"'s mu is a meanlog.

"THT" is not an independent convolution. Tancredi's (2002) composed error is skew-tt, a scale mixture in which vv and uu are divided by the same V/a\sqrt{V/a}. Treating it as tt noise plus an independent half-normal – which is what "tHN" actually is – gets the log density wrong by up to 1.74, so the two models take different paths here despite looking alike.

Verified two ways: against pcomposed for the half-normal case, agreeing to 3.5×10−153.5\times10^{-15} relative including a lower tail at log⁡F=−209\log F = -209; and for all thirteen models against a four-million-draw simulation, with a maximum absolute error of 4×10−44\times10^{-4} against a Monte Carlo standard error of 7.5×10−47.5\times10^{-4}. Numerically differentiating this CDF reproduces each model's own stored log-density to about 10−810^{-8}, except for the simulated-ML models "NLN" and "NW", where the difference is the simulation error of their likelihood rather than a disagreement about the model.

Value

A numeric vector of probabilities (or their logs).

See Also

pcomposed, dcomposed, sfm

Examples

p <- c(sigv = 0.5, sigu = 1.1)
pcomposed_model(c(-2, -1, 0, 1), "NE", p)

d <- as.data.frame(data_gen_cs(N = 200, rand = 5, sig_u = 1, sig_v = 0.5,
                               cons = 0.5, beta1 = 0.5, beta2 = 0.5,
                               a = 5, mu = 0.1))
f <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d)
head(composed_cdf(f, data = d))

Pitt and Lee (1981) Model III: the multivariate truncated normal panel likelihood

Description

Notes on psfm(model_name = "PL80_MVTN"), which fits the likelihood derived in Appendix 2 of Pitt and Lee (1981) but never used there.

Details

The model is

yit=xit′β+uit+vit,uit≤0,y_{it} = x_{it}'\beta + u_{it} + v_{it}, \qquad u_{it} \le 0,

with the firm's inefficiency vector ui=(ui1,…,uiT)′u_i = (u_{i1},\dots,u_{iT})' drawn from a TT-variate normal N(0,Σ)N(0,\Sigma) truncated to the negative orthant, and vitv_{it} iid N(0,σv2)N(0,\sigma_v^2) independent of uu. Unlike model_name = "PL80", which holds inefficiency fixed over time, here it varies across periods and is correlated within a firm – Σ\Sigma carries that dependence.

Why it was never used. Pitt and Lee derived this likelihood and then set it aside, writing that it “is difficult to evaluate since the quantities P0P_0 and P(yi−xiβ)P(y_i - x_i\beta) involve TT-dimensional numerical integrals”, and estimating Model III by Zellner seemingly-unrelated regression instead. Those quantities are orthant probabilities of a multivariate normal. mnormt::sadmvn() evaluates one in about 3 milliseconds at T=6T = 6, so a likelihood evaluation costs roughly N+1N+1 of them – about 0.3 s at N=100N = 100, and about 12 s for a whole fit at N=80N = 80, T=4T = 4. What was intractable in 1981 is merely slow now.

Writing Q−1=Σ−1+I/σv2Q^{-1} = \Sigma^{-1} + I/\sigma_v^2 and μi=Qεi/σv2\mu_i = Q\varepsilon_i/\sigma_v^2, the per-firm log density is

log⁡f(εi)=−T2log⁡2π−Tlog⁡σv−12log⁡∣Σ∣+12log⁡∣Q∣−log⁡P0−12(εi′εiσv2−εi′Qεiσv4)+log⁡Pi,\log f(\varepsilon_i) = -\tfrac{T}{2}\log 2\pi - T\log\sigma_v - \tfrac{1}{2}\log|\Sigma| + \tfrac{1}{2}\log|Q| - \log P_0 - \tfrac{1}{2}\left(\tfrac{\varepsilon_i'\varepsilon_i}{\sigma_v^2} - \tfrac{\varepsilon_i'Q\varepsilon_i}{\sigma_v^4}\right) + \log P_i,

where P0=Pr⁡(w≤0)P_0 = \Pr(w \le 0) for w∼N(0,Σ)w \sim N(0,\Sigma) is the truncation constant and Pi=Pr⁡(w≤0)P_i = \Pr(w \le 0) for w∼N(μi,Q)w \sim N(\mu_i, Q).

Σ\Sigma is equicorrelated, not unrestricted. It is parameterized as Σ=σu2[(1−ρ)I+ρ11′]\Sigma = \sigma_u^2[(1-\rho)I + \rho \mathbf{1}\mathbf{1}'], costing two parameters. An unrestricted Σ\Sigma costs T(T+1)/2T(T+1)/2 – 21 at T=6T=6, 55 at T=10T=10 – on top of β\beta and σv\sigma_v, every one identified only through orthant probabilities. The equicorrelated form captures what the general Σ\Sigma was introduced for: dependence of a firm's inefficiency across periods. ρ=0\rho = 0 gives inefficiency independent over time; ρ→1\rho \to 1 approaches the time-invariant "PL80" case. Because the form is equicorrelated every matrix quantity above is closed form (Sherman-Morrison), so only the orthant probabilities are numerical.

Requirements and limits. A balanced panel with T≥2T \ge 2: Σ\Sigma is a single T×TT \times T matrix shared by every firm, and with T=1T = 1 there is no cross-period dependence for it to describe. An unbalanced panel is an error pointing at "PL80". Reported parameters are sigv, sigu, rho and the frontier coefficients.

u_hat is the posterior mean of uu from the Gaussian part, floored at zero, not E[uit∣εi]E[u_{it}\mid\varepsilon_i]: the exact conditional mean of a truncated multivariate normal is another TT-dimensional integral. It is exact where the truncation does not bind and an approximation where it does.

Author(s)

David H. Bernstein

References

Pitt, M.M. and Lee, L.-F. (1981) 'The measurement and sources of technical inefficiency in the Indonesian weaving industry', Journal of Development Economics, 9(1), pp. 43-64. See Appendix 2 for the likelihood.

See Also

psfm, data_gen_p for the y_pl_mvtn column that tests it.

Examples

library(sfa)
d <- data_gen_p(t = 4, N = 60, rand = 5, sig_u = 1, sig_v = 0.3, sig_r = 0.2,
                sig_h = 0.4, cons = 0.5, beta1 = 0.5, beta2 = 0.5)
f <- psfm(y_pl_mvtn ~ x1 + x2, model_name = "PL80_MVTN",
          data = as.data.frame(d), individual = "name")
f$out

Predictions, fitted values and residuals for stochastic frontier fits

Description

Standard modelling generics for objects of class "sfareg", as returned by sfm, psfm, zsfm and ttsfm.

Usage

## S3 method for class 'sfareg'
predict(object, newdata = NULL,
        type = c("frontier", "response", "efficiency"), ...)
## S3 method for class 'sfareg'
fitted(object, ...)
## S3 method for class 'sfareg'
residuals(object, ...)

Arguments

object

An object of class "sfareg".

newdata

Optional data frame containing the frontier regressors at which to predict. If omitted, the data the model was fitted to is used.

type

"frontier" (default) returns the deterministic frontier x′βx'\beta. "response" shifts it by the predicted inefficiency, giving an estimate of E[y∣x]E[y \mid x] rather than of the frontier itself, with the sign following the production/cost convention the model was fitted under. "efficiency" returns the predicted technical efficiency E[exp⁡(−u)∣ε]E[\exp(-u) \mid \varepsilon].

...

Ignored.

Details

Only the first part of a pipe formula (y ~ x | z | zp) describes the frontier; later parts parameterize variances and take no part in prediction.

Frontier coefficients are identified by matching the fitted coefficient names against the frontier design matrix, so these methods work for every model in the package without depending on where a given model happens to place its auxiliary parameters (lambda/sigma for NHN, sigv/sigu for NE, sigmaSq/gamma for PL80, and so on).

type = "efficiency" and type = "response" are available only for the estimation sample, since both require the composed residual and therefore the response; requesting them with newdata raises an error rather than returning something misleading. "efficiency" additionally requires a model that returns an efficiency prediction – see sfm for which ones do.

Value

A numeric vector, one element per observation.

Author(s)

David H. Bernstein

See Also

sfm, psfm, sfareg-methods

Examples

library(sfa)
d <- data_gen_cs(N = 200, rand = 1, sig_u = 0.8, sig_v = 0.3,
                 cons = 1, beta1 = 0.5, beta2 = 0.5, a = 5, mu = 0.5)
f <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d)

head(fitted(f))
head(residuals(f))
head(predict(f, type = "efficiency"))
head(predict(f, newdata = d[1:5, ]))

Print method for "sfareg" objects

Description

Compact display of a stochastic frontier fit, as returned by sfm, psfm, zsfm or ttsfm.

Usage

## S3 method for class 'sfareg'
print(x, ...)

Arguments

x

An object of class "sfareg", as returned by sfm, psfm, zsfm or ttsfm.

...

Additional arguments passed to other methods

Details

Prints the model name, the estimated coefficients with their standard errors and t-values (the out matrix), and the log-likelihood where the estimator has one. For a fuller display, including the frontier and variance-parameter blocks, use summary.sfareg.

Value

No return value, called for side effects

Author(s)

David H. Bernstein

See Also

summary.sfareg, sfareg-methods

Examples

library(sfa)     

cs_data_trial   <- data_gen_cs(N= 1000, rand   = 1,  sig_u  = 0.3, sig_v  = 0.3, 
cons   = 0.5,       beta1  = 0.5,   beta2  = 0.5, a      = 4, mu     = 1)

cs.nhnz     <-  sfm(formula    = y_pcs_z ~ x1 +x2| z,    model_name = "NHN_Z",
                    data       = cs_data_trial,          PSopt      = TRUE)
print(cs.nhnz)

Panel Stochastic Frontier Models

Description

Function to implement various panel data stochastic frontier estimators

Usage

psfm(formula, model_name = c("TRE_Z", "GTRE_Z", "TRE",
                    "GTRE", "GTRE_FML", "TFE", "TFE_WMLE", "FD",
                    "GTRE_SEQ1", "GTRE_SEQ2", "SSFE",
                    "PL80", "PL80_MVTN", "BC92", "K1990", "K1990modified",
                    "CSS", "LS", "SSRE", "SSCRE", "KSS"), data,
                    maxit.bobyqa = 5000, maxit.nlminb = 500,
                    maxit.psoptim = 100, maxit.optim = 1000,
                    REPORT = 1, trace = 3, pgtol = 0, individual,
                    halton_num = NULL, start_val = FALSE, gamma = FALSE,
                    PSopt = FALSE, optHessian = TRUE, inefdec = TRUE, Method = "L-BFGS-B",
                    verbose = FALSE, rand.gtre = NULL, rand.psoptim = NULL,
                    OPG_calc = FALSE,
                    estimator = c("fiml", "sml", "seq1", "seq2"),
                    collinear_action = c("start_only", "error", "warn_drop"),
                    time = NULL, tfe_lambda_max = 100,
                    keep_objective = FALSE, kss_L = "auto",
                    kss_smooth = "auto", kss_L_max = 7L, mundlak = NULL,
                    sim_type = c("halton", "sobol", "torus", "uniform"),
                    antithetics = FALSE, sim_burn = NULL,
                    sim_scrambling = 0L, sim_prime = NULL)

Arguments

formula

a symbolic description for the model to be estimated

model_name

Which panel estimator to fit. All are described in ‘Details’; in brief:

"TRE", "TRE_Z"

Greene's (2005) true random effects, by simulated maximum likelihood. _Z adds determinants of σu\sigma_u.

"GTRE", "GTRE_Z"

the four-component model separating persistent from transient inefficiency, by simulated maximum likelihood. _Z adds determinants.

"GTRE_FML"

the same four-component model by full information maximum likelihood through its closed-skew-normal representation; deterministic, but requires a balanced panel.

"GTRE_SEQ1", "GTRE_SEQ2"

two-step/moment-based estimators of the four-component model. Not maximum likelihood – no logLik().

"TFE"

Greene's (2005) true fixed effects. Changed meaning in 1.1.3 – see ‘Details’.

"TFE_WMLE"

Chen, Schmidt and Wang's (2014) within maximum-likelihood estimator.

"FD"

the first-difference estimator of Wang and Ho (2010).

"SSFE"

Schmidt and Sickles' (1984) fixed-effects estimator, inefficiency read off the firm effects. Not maximum likelihood – no logLik().

"CSS"

Cornwell, Schmidt and Sickles (1990): a firm-specific quadratic in time, so inefficiency varies over time and firms may cross over. Not maximum likelihood – no logLik().

"LS"

Lee and Schmidt (1993): one common temporal pattern scaled per firm, αit=δtαi\alpha_{it} = \delta_t \alpha_i. Time-varying but no crossover. Not maximum likelihood – no logLik().

"SSRE"

Schmidt and Sickles (1984) random effects: the GLS estimator, with inefficiency read off the firm effects' BLUPs. More efficient than "SSFE" and identifies time-invariant regressors, at the cost of assuming the effects are uncorrelated with the regressors. Not maximum likelihood – no logLik().

"SSCRE"

Correlated random effects: "SSRE" augmented with the within-firm means of the time-varying regressors (Mundlak 1978), so that correlation between effects and regressors is modelled rather than assumed away. Not maximum likelihood – no logLik().

"KSS"

Kneip, Sickles and Song (2012): the firm effect is a smooth function of time lying in an LL-dimensional space whose basis is estimated from the data. Nests "SSFE", "LS" and "CSS". Balanced panels only. Not maximum likelihood – no logLik().

"PL80"

Pitt and Lee (1980): time-invariant inefficiency.

"BC92"

Battese and Coelli (1992): exponential time decay.

"PL80_MVTN"

Pitt and Lee's (1981) Model III: inefficiency varies over time and is CORRELATED within a firm, through a multivariate normal truncated to the negative orthant. This is the likelihood from their Appendix 2, which they derived but did not use, calling the T-dimensional integrals intractable. Balanced panels only. See PL80_MVTN.

"K1990", "K1990modified"

Kumbhakar (1990) time patterns for inefficiency.

Matching ignores case, so "tre", "TRE" and "Tre" are the same choice.

data

A panel data set. This may be an ordinary data.frame (or tibble/data.table) – the panel index is constructed internally from the individual and time arguments – or a plm::pdata.frame that already carries an index, which is used as-is.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine

maxit.nlminb

Maximum number of iterations for the nlminb optimization routine.

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine

maxit.optim

Maximum number of iterations for the optim optimization routine

REPORT, trace, pgtol

Accepted but not used by psfm(): the optimizer's own tracing is controlled by verbose.

individual

individual unit in the regression model

halton_num

Number of simulation draws per firm for the simulated-ML models ("GTRE" with estimator = "sml", "TRE", "GTRE_Z", "TRE_Z"). Despite the name, it counts draws of whichever sequence sim_type selects. NULL uses ⌈n⌉+100\lceil\sqrt{n}\rceil + 100.

start_val

starting value (optional)

gamma

Logical. Report the variance parameters in the γ=σu2/(σu2+σv2)\gamma = \sigma_u^2/(\sigma_u^2 + \sigma_v^2) parameterization instead of the default. Defaults to FALSE.

PSopt

use psoptim optimization routine (T or F)

optHessian

Logical. Should a numerically differentiated Hessian matrix be returned while using the optim routine? (for optim routine)

inefdec

Production or cost function

Method

Method passed to optim for the final stage. Must be "L-BFGS-B": it is the only optim method that applies the lower and upper bounds this package uses to keep scale parameters positive, and any other value is refused with an error rather than quietly optimized without them.

verbose

Logical. Print optimization progress messages? Default is FALSE.

rand.psoptim

Integer. Seed for replication of psoptim. Default to NULL.

rand.gtre

Integer. Seed for replication of the gtre model. Default to NULL.

OPG_calc

Logical. Should the outer product of gradients be calculated for inference. Default to FALSE.

estimator

How to estimate the four-component GTRE model, in the same spirit as sfm's estimator: one model, several routes to it. Applies to model_name = "GTRE" only, and is ignored with a warning elsewhere.

"fiml"

the default. Full information maximum likelihood through the closed-skew-normal representation. Deterministic – nothing is integrated by Monte Carlo, so repeated fits to the same data agree exactly. Requires a balanced panel.

"sml"

Simulated maximum likelihood over Halton draws. Handles unbalanced panels. This is what model_name = "GTRE" meant through version 1.1.3.

"seq1", "seq2"

The two-step moment-based estimators. Not maximum likelihood, so they carry no logLik().

The older names remain and are unchanged: "GTRE_FML", "GTRE_SEQ1" and "GTRE_SEQ2" select the same three routes directly.

On an unbalanced panel "fiml" cannot be fitted. If you simply took the default, psfm() warns and falls back to "sml" – erroring would make "GTRE" unusable by default on a whole class of data. If you asked for "fiml" explicitly it errors instead, rather than silently fitting a different estimator than the one you named.

collinear_action

One of "start_only" (default), "error", or "warn_drop". Controls what happens when the between-individual design matrix used to build starting values is rank deficient. This is checked separately from the pooled design because the two can differ: time dummies, for example, are routinely estimable in a pooled specification while collapsing onto the intercept once averaged within each unit, which is exactly the situation that makes the internal random-effects initialization fail. "start_only" keeps the model you asked for and removes the offending columns from the starting-value regression only, taking their starting values from a pooled OLS fit; "error" stops before estimation and names the offending columns; "warn_drop" removes those terms from the estimated model. All three report what was found. Only relevant to the models initialized by a random-effects regression (GTRE, TRE and their _Z/SEQ variants).

time

Character. Name of a time-period column in data, used by the error-components frontier models ("PL80", "BC92", "K1990", "K1990modified"). If NULL (the default), a sequential within-individual time index is constructed from row order, which assumes data is already sorted chronologically within each individual.

tfe_lambda_max

Numeric. Upper bound placed on λ=σu/σv\lambda = \sigma_u/\sigma_v when fitting model_name = "TFE"; ignored by every other model. Defaults to 100, i.e. σv\sigma_v is not allowed below about 1% of σ\sigma. This is a guard against a genuine degeneracy of Greene's likelihood rather than a convergence tolerance – see ‘Details’. A fit that pins at the bound warns and should not be read as an estimate of λ\lambda.

keep_objective

Retain the likelihood closure on the fitted object as $objective, for every model_name fit by maximum likelihood: the simulated-ML models "GTRE", "TRE", "GTRE_Z" and "TRE_Z", and, from 1.2.1, the closed-form likelihoods "GTRE_FML", "TFE_WMLE", "FD", "PL80", "PL80_MVTN", "BC92", "K1990" and "K1990modified". This lets the log-likelihood be evaluated away from the optimum, which is what profiling a single parameter requires; sfa_diagnostics uses it, and it is how the sigh convergence question was localized (see ‘Details’). FALSE by default, since the closure captures the data and the draws and so is not small. From 1.2.1 the closure also accepts per_obs = TRUE, returning one log-likelihood contribution per firm – firms are the independent units in these models – so the score-based tools work on these fits: influence_sfa, vcov(type = "bhhh"), the sandwich methods, TIC and vuong. Their rows and their sandwich standard errors are therefore firm-level, i.e. clustered by firm. The remaining model_names ("TFE", "SSFE", "SSRE", "SSCRE", "CSS", "LS", "KSS", "GTRE_SEQ1" and "GTRE_SEQ2") are moment-based, FE/LSDV or sequential rather than maximum likelihood, so they have no score matrix; for those the argument has no effect and psfm() warns.

kss_L

"KSS" only. The dimension LL of the factor space. "auto" (the default) selects it by the Bai and Ng (2002) ICp2IC_{p2} criterion; give an integer to fix it.

kss_smooth

"KSS" only. The smoothing penalty applied to each firm's residual trajectory before the basis is extracted. "auto" (the default) chooses it by generalized cross-validation; 0 turns smoothing off, which reduces the model to an unrestricted factor model.

kss_L_max

"KSS" only. Largest LL the automatic criterion may choose. The criterion is capped internally at ⌊T/2⌋\lfloor T/2 \rfloor as well, for the reason given in ‘Details’, and warns if it selects that cap. An explicit kss_L is not subject to the ⌊T/2⌋\lfloor T/2 \rfloor cap, only to min⁡(T−1,n−1)\min(T-1, n-1).

sim_type

Sequence used to build the draws for the simulated-ML models listed under halton_num; ignored by every other model. "halton" (default) reproduces previous behaviour exactly. "sobol" accepts sim_scrambling; "torus" accepts sim_prime; "uniform" is pseudorandom, a baseline rather than a recommendation. The same options as sfm, built by the same code, so see ‘Details’ there. Each firm receives its own block of the sequence, and rand.gtre randomizes it by a shift modulo 1 (by the seed, for "sobol" and "uniform").

antithetics

Logical. If TRUE, half of each firm's draws are taken from the sequence and the other half created as their mirror images. Default FALSE.

sim_burn

Number of leading elements of the sequence discarded. Default NULL, meaning 1000.

sim_scrambling

Integer 0–3, sim_type = "sobol" only, passed to randtoolbox::sobol.

sim_prime

Prime for sim_type = "torus". Default NULL uses the generator's own choice.

mundlak

Mundlak adjustment terms: a formula naming the regressors whose within-firm means should be added to the frontier, or ~ . for every frontier regressor. NULL (the default) adds nothing. Only for models that treat the firm effect as random; "TFE", "TFE_WMLE", "FD" and "SSFE" refuse it, because they have already removed the effect. See ‘Details’.

Details

The generalized true random effects model (GTRE, 4-component model) and true random effects models (TRE) are both estimated by simulated maximum likelihood based on the paper by the Fillipini and Greene (2016, JPA). The TRE_Z and GTRE_Z allow for modeling the u-component of the GTRE and TRE with determinants of inefficiency. The first-difference estimator (FD) of Wang and Ho (2010, JoE) is also available.

model_name what it models use it when
"TRE", "TRE_Z" true random effects: firm heterogeneity plus transient inefficiency firm effects are not inefficiency
"GTRE", "GTRE_Z" four components: heterogeneity, persistent and transient inefficiency both kinds of inefficiency matter
"GTRE_SEQ1", "GTRE_SEQ2" the four-component model in two steps TT is large; inconsistent for fixed TT
"TFE", "TFE_WMLE" true fixed effects firm effects may be correlated with the regressors
"FD" first differences (Wang and Ho 2010) fixed effects with inefficiency determinants
"PL80", "BC92", "PL80_MVTN" time-invariant, decaying or correlated inefficiency the classical panel frontiers
"K1990", "K1990modified" Kumbhakar (1990) time paths for inefficiency inefficiency follows a smooth time pattern
"SSFE", "SSRE", "SSCRE" Schmidt-Sickles: the firm effect is the inefficiency no distributional assumption on uu
"CSS", "LS", "KSS" distribution-free, time-varying firm effects inefficiency changes over time, no distribution

Each family has its own subsection below.

The four-component model (GTRE)

Choosing how GTRE is estimated. Since version 1.1.4 the four routes to the four-component model are selected with estimator rather than by picking among four model_name values that looked like four different models. model_name = "GTRE" now defaults to estimator = "fiml"; through 1.1.3 it fitted the simulated-ML estimator, which is estimator = "sml". Scripts that pass "GTRE" therefore get a different estimator than they did and are warned once per call. The explicit names "GTRE_FML", "GTRE_SEQ1" and "GTRE_SEQ2" are unaffected.

Two ways of estimating the same four-component model. "GTRE" and "GTRE_FML" both fit

yit=β0+xit′β+ri−hi+vit−uit,y_{it} = \beta_0 + x_{it}'\beta + r_i - h_i + v_{it} - u_{it},

with ri,vitr_i, v_{it} two-sided and hi,uith_i, u_{it} one-sided (half-normal), separating persistent inefficiency hih_i from transient inefficiency uitu_{it}. They differ in how the likelihood is obtained.

"GTRE" uses simulated maximum likelihood over Halton draws. "GTRE_FML" uses full information maximum likelihood through the model's closed-skew-normal representation: with A=−[1T,IT]A = -[1_T, I_T], V=diag(σh2,σu2IT)V = \mathrm{diag}(\sigma_h^2, \sigma_u^2 I_T) and Σ=σv2IT+σr21T1T′\Sigma = \sigma_v^2 I_T + \sigma_r^2 1_T 1_T', the firm-level density is available in closed form and nothing is integrated by Monte Carlo. The likelihood is therefore deterministic, and repeated fits to the same data give identical answers.

The (T+1)(T+1)-dimensional normal CDF this requires would normally be the bottleneck, but its covariance is diagonal-plus-rank-one for this model, so it collapses to a one-dimensional Gauss-Hermite integral – the same reduction used for "TFE".

"GTRE_FML" reports raw standard deviations sigr, sigv, sigh, sigu (individual effect, noise, persistent inefficiency, transient inefficiency), not the λ\lambda/σ\sigma reparameterization "GTRE" uses, and it requires a balanced panel – the closed-skew-normal pieces are built at a single TT. It errors rather than silently mis-estimating if the panel is unbalanced; use "GTRE" in that case.

Starting values for "GTRE_FML". The closed-skew-normal likelihood carries a boundary optimum at σh=0\sigma_h = 0, where the model collapses to "TRE" and the intercept absorbs the missing E[h]=σh2/πE[h] = \sigma_h\sqrt{2/\pi}. Two different things can put a fit there, and the likelihood tells them apart. A fit whose boundary solution has a lower likelihood than the true parameter vector reached it by getting stuck, and is an optimizer failure; one whose boundary solution has a higher likelihood reached it because that is where the maximum is, and is the correct answer for that sample (see ‘When a persistent scale collapses to zero’ below). The start search addresses only the first kind. Following Colombi (2010) and Colombi, Martini and Vittadini (2011), who recommend seeding this likelihood from the two-step moment estimates, psfm() builds two candidate starts – the random-effects decomposition shared with "GTRE", and the two-step estimator reported by "GTRE_SEQ2" – evaluates the likelihood at each, and begins from the better one. Which was chosen, and the log-likelihood at both, is returned in start_search. Supplying start_val overrides the search.

Note that σr\sigma_r and σh\sigma_h enter the density only through σr2+σh2\sigma_r^2 + \sigma_h^2; their separation rests entirely on the skewness term. That pair is therefore the least precisely estimated part of the model even when the fit is at the global optimum, which is consistent with the confounding between latent heterogeneity and persistent inefficiency reported in the source papers.

"GTRE_SEQ1" and "GTRE_SEQ2" are large-T estimators. Both take the random-effects decomposition from plm and hand its two output series to a second stage – "GTRE_SEQ1" fits an intercept-only normal–half-normal likelihood to each, "GTRE_SEQ2" inverts their second and third moments. The second stage treats those series as if they were draws from vit−uitv_{it}-u_{it} and ri−hir_i-h_i, and at finite TT they are not: alpha_hat is the shrunken predictor c (αi+eˉi)c\,(\alpha_i+\bar e_i) with c=Tσα2/(Tσα2+σϵ2)c = T\sigma_\alpha^2/(T\sigma_\alpha^2+\sigma_\epsilon^2), and eps_hat is the quasi-demeaned residual (1−θ)αi+eit−θeˉi(1-\theta)\alpha_i + e_{it} - \theta\bar e_i with θ=1−σϵ/Tσα2+σϵ2\theta = 1-\sigma_\epsilon/\sqrt{T\sigma_\alpha^2+\sigma_\epsilon^2}.

Relative to the latent draws this attenuates the third central moment on the ϵ\epsilon side by (1−θ/T)3+(T−1)(−θ/T)3(1-\theta/T)^3+(T-1)(-\theta/T)^3 (about 0.81 at T=10T=10) and scales the α\alpha side by c2c^2 and c3c^3. None of these factors tends to 1 as NN grows with TT fixed, so both estimators converge to the wrong constants and are inconsistent for fixed TT; each call warns accordingly. On the package's own convergence design (T=10T=10, σh=0.8\sigma_h=0.8, σr=0.4\sigma_r=0.4) the estimates settle at γuv≈0.87\gamma_{uv}\approx0.87 against 0.917 and σhr2≈0.68\sigma^2_{hr}\approx0.68 against 0.80, and the log MSE-on-log NN slopes are near zero rather than −1-1. The same routines applied to the latent draws themselves recover the truth at the N\sqrt{N} rate, which locates the problem in the first-stage decomposition rather than in either second stage. Raising TT removes the bias; raising NN only sharpens it. For a consistent alternative use model_name = "GTRE", which estimates all four variance components jointly.

The standard errors these two report are delta-method errors built from the iid moment variances, so they describe sampling variability around each estimator's probability limit, not around the truth. They are also optimistic on the ϵ\epsilon side, whose nn residuals are correlated within firm.

When a persistent scale collapses to zero

When a persistent scale collapses to zero. GTRE carries two persistent components – σh\sigma_h, persistent inefficiency, and σr\sigma_r, the firm random effect – and the likelihood cannot always separate them in a given sample. When it cannot, it merges them: one scale goes to zero and the other absorbs its variation, coming back inflated.

This is common rather than exceptional. On simulated data at T=6T = 6 with both components genuinely present (σr=0.20\sigma_r = 0.20, σh=0.40\sigma_h = 0.40), one of the two collapsed in 37% of 87 replications, and the commoner direction was σr→0\sigma_r \to 0 rather than σh→0\sigma_h \to 0. The share with both interior does improve with NN (15, 19 and 21 of about 29 at N=50,100,200N = 50, 100, 200) but slowly.

A collapse is usually the right answer, not a numerical failure. On one such replication – whose realized σh\sigma_h scale was 0.3695, so persistent inefficiency was certainly present – the boundary solution had a log-likelihood higher than the true parameter vector by 3.66, a likelihood ratio of about 39. This is the panel counterpart of the cross-sectional wrong-skewness result (Waldman 1982), where σu=0\sigma_u = 0 is the correct maximum likelihood estimate rather than an optimizer failure; see skewness_test. This is not in tension with the start search described above under "GTRE_FML": a boundary fit that beats the truth on likelihood is the estimate, while one that loses to it is a trapped optimizer, and the search exists to remove the second so that what remains is the first.

What a collapse does to the rest of the fit, exactly. One rule covers it: a collapse conserves the persistent variance and relocates the persistent mean into the intercept. Those are two different destinations. hh is half-normal, so it carries both a mean E[h]=σh2/πE[h] = \sigma_h\sqrt{2/\pi} and a variance Var(h)=σh2(1−2/π)\mathrm{Var}(h) = \sigma_h^2(1 - 2/\pi), and the model has a separate place for each: the frontier intercept takes the mean, since E[y]=β0−E[h]−E[u]E[y] = \beta_0 - E[h] - E[u], and the surviving persistent scale takes the variance. The intuitive shortcut – that the survivor comes back at σh2+σr2\sqrt{\sigma_h^2 + \sigma_r^2} – ignores that split and is wrong.

surviving scale intercept
σh→0\sigma_h \to 0 σ^r=σr2+σh2(1−2/π)\hat\sigma_r = \sqrt{\sigma_r^2 + \sigma_h^2(1-2/\pi)} shifts by −E[h]-E[h]
σr→0\sigma_r \to 0 σ^h=σh2+σr2/(1−2/π)\hat\sigma_h = \sqrt{\sigma_h^2 + \sigma_r^2/(1-2/\pi)} shifts by +ΔE[h]+\Delta E[h]

At σh=0.40\sigma_h = 0.40, σr=0.20\sigma_r = 0.20 that predicts 0.3133 and −0.3192-0.3192 in the first row, 0.5197 and +0.0955+0.0955 in the second, with nothing fitted. Over 1000 replications at each of five sample sizes the measurements are 0.2973, −0.3035-0.3035, 0.5058 and +0.0931+0.0931 – within 2.5% to 5.1%, and all four slightly below prediction, which is what selection produces rather than a wrong constant.

The practical consequence is that a collapsed fit does not merely mislabel the persistent split. It displaces the intercept by a known and substantial amount, so any level read off coef() is displaced with it.

What a collapse is telling you about your data. It is wrong skewness, measured between firms rather than within them. Take the OLS residuals, average them within each firm – that average is the persistent part of the error – and look at their skew. Over 1000 replications, the samples whose between-firm skew came out positive (the wrong way for a production frontier) lost σh\sigma_h to the boundary 69% of the time, against 3.6% for the rest; the area under the ROC curve for that single statistic is 0.955. Not one wrong-skew sample collapsed σr\sigma_r instead. So a collapsed σh\sigma_h is not a numerical accident to be retried with different starting values: this sample carries no between-firm evidence of persistent inefficiency, and the estimate is reporting that.

Note that this is a reading of the sign, not a test. The firm mean of the residuals contains −uˉi-\bar u_i as well as −hi-h_i, and the mean of TT half-normals is itself skewed the same way, so on short panels the between-firm skew leans negative even when σh=0\sigma_h = 0 – 82% of the time at T=4T = 4, 63% at T=10T = 10, 53% at T=30T = 30. skewness_test is therefore not applicable to the between-firm residuals as it stands.

Whether firms or periods help depends on the estimator, and the two answers are opposite. Crossed (N,T)(N, T) designs on the same data-generating process, fitting the collapse rate on log⁡N\log N and log⁡T\log T:

estimator fits log N log T
"fiml" (the default) 600 -1.66 (0.18) -0.93 (0.20)
"sml" 251 -0.68 (0.26) -1.07 (0.27)

Under the default, firms buy about 1.8 times what periods do; under simulated ML it is the other way round, periods buying about 1.6 times what firms do. Restricting the "fiml" design to N≤200N \le 200, cell for cell the same grid the "sml" one used, leaves it at −1.64-1.64 (0.22)(0.22), so this is the estimator and not the design; the difference in the NN coefficient is significant (z=2.8)(z = 2.8) and the difference in the TT coefficient is not.

The practical reading, for the estimator you get by default: add firms. At T=4T = 4 the collapse rate falls 68%, 36%, 18%, 6% across N=50,100,200,400N = 50, 100, 200, 400, and reaches 0% at N=400,T=16N = 400, T = 16. Under "sml" the same row is flat – 60%, 39%, 41% – which is why a short panel really is a dead end there however many firms it has. If you must use estimator = "sml" (an unbalanced panel forces it), then length is what buys the split, and a four-period panel will not give it to you.

How to tell in advance whether your panel can support the split. One quantity governs it, and it is not σh/σr\sigma_h/\sigma_r. What the likelihood has to do is read the persistent part out of firm means that the transient block contaminates, so write the firm-mean composite as ri−hi−uˉi+vˉir_i - h_i - \bar u_i + \bar v_i and compare its persistent variance with the transient variance left in it:

S/N=σr2+σh2(1−2/π)[σu2(1−2/π)+σv2]/TS/N = \frac{\sigma_r^2 + \sigma_h^2(1 - 2/\pi)}{[\sigma_u^2(1 - 2/\pi) + \sigma_v^2]/T}

Across six designs varying all four scales in different combinations (50 replications each, N=100N = 100, T=6T = 6), this orders the collapse rate perfectly – Spearman −1.000-1.000 under both estimators, and a logistic coefficient on log⁡(S/N)\log(S/N) of −1.15-1.15 (0.20)(0.20):

S/N collapse σh→0\sigma_h \to 0 σr→0\sigma_r \to 0
0.67 54% 0.14 0.40
0.90 50% 0.10 0.40
0.96 40% 0.21 0.19
1.30 31% 0.15 0.17
5.20 8% 0.06 0.02
11.69 6% 0.04 0.02

As a rule of thumb: below S/N≈1S/N \approx 1 about half your fits will merge the two components, and it takes roughly S/N>5S/N > 5 to get under 10%.

Two things follow that are easy to get backwards. First, raising σh/σr\sigma_h/\sigma_r does not help – it only decides which scale dies. Going from a ratio of 2 to 4 by shrinking σr\sigma_r takes σh→0\sigma_h \to 0 down (0.15 to 0.10) and σr→0\sigma_r \to 0 up (0.17 to 0.40), and the net rate rises from 31% to 50%, because shrinking σr\sigma_r removes persistent variance and lowers S/NS/N from 1.30 to 0.90. Holding the ratio at 2 and scaling both components instead takes the rate from 31% to 8%. Second, more transient inefficiency hurts: σu\sigma_u enters the denominator, so a noisier transient block contaminates the firm means and makes the persistent split harder.

Note what this quantity does and does not explain. TT enters it directly, so it covers the contamination channel by which longer panels help. It says nothing about NN, which helps through a different route – sampling error in what is effectively a deconvolution from NN firm-level draws – which is why the two dimensions have to be measured separately, as above.

One limit on all of that, stated because it is easy to over-read. Those elasticities were measured at σr=0.20\sigma_r = 0.20 against σh=0.40\sigma_h = 0.40, where persistent inefficiency is the larger of the two time-invariant components. On a design where the random effect dominates instead (σr=0.50\sigma_r = 0.50, σh=0.40\sigma_h = 0.40, N=100N = 100, T=6T = 6) the level ordering reverses: "fiml" collapses on 33% of replications and "sml" on 20%. Which estimator handles the split better is not a fact about the estimators alone – it depends on how much persistent inefficiency there is to find.

psfm therefore reports the condition rather than preventing it, in $sigh_at_bound and $sigr_at_bound and in a warning naming the surviving scale. Bounding either parameter away from zero would corrupt precisely the samples where the boundary is the answer. What the warning asks of you is to read the two persistent scales together and to treat the split between them as unidentified in that sample – their sum remains informative even where the division does not.

True fixed effects

The two true-fixed-effects estimators, and a change of meaning in "TFE". Both "TFE" and "TFE_WMLE" fit the same model,

yit=αi+xit′β+vit−uit,v∼N(0,σv2), u∼N+(0,σu2),y_{it} = \alpha_i + x_{it}'\beta + v_{it} - u_{it},\quad v \sim N(0,\sigma_v^2),\ u \sim N^+(0,\sigma_u^2),

with αi\alpha_i an unrestricted individual effect; they differ in how they estimate it.

"TFE" is Greene's (2005, JoE/JPA) true fixed effects estimator: the ordinary composed-error likelihood with the NN individual intercepts estimated jointly with (λ,σ,β)(\lambda, \sigma, \beta). This package concentrates the αi\alpha_i out at every likelihood evaluation (each is the unique root of a strictly decreasing score, solved by safeguarded Newton to machine precision), so the reported parameter vector, its standard errors and the out layout are the same 2 + K as "TFE_WMLE". It is the estimator subject to the incidental parameters problem: with TT fixed, σu\sigma_u is biased upward and efficiency scores correspondingly pessimistic.

"TFE_WMLE" is Chen, Schmidt and Wang's (2014, JoE) within maximum-likelihood estimator, which applies maximum likelihood to the deviations from individual means. The αi\alpha_i drop out of the likelihood entirely, so the estimator is free of the incidental parameters problem; the firm effects are recovered afterwards from the mean residual with the usual 2/π σu\sqrt{2/\pi}\,\sigma_u adjustment.

The two are not nested and their log-likelihoods are not on the same scale – "TFE" evaluates the likelihood of the data, "TFE_WMLE" that of the within-transformed deviations – so logLik(), AIC() and BIC() must not be used to choose between them.

Through sfa 1.1.2, model_name = "TFE" selected the Chen-Schmidt-Wang estimator. It now selects Greene's, matching standard usage; the Chen-Schmidt-Wang estimator is "TFE_WMLE". Existing scripts that pass "TFE" therefore get a different estimator than they did, and are warned once per call.

Degeneracy of the Greene likelihood. Because αi\alpha_i is unrestricted, Greene's likelihood always has a supremum on the σv→0\sigma_v \to 0 boundary: setting αi=max⁡t(yit−xit′β)\alpha_i = \max_t (y_{it} - x_{it}'\beta) makes every composed error weakly negative, Φ(−λe/σ)→1\Phi(-\lambda e/\sigma) \to 1, and what remains is the deterministic-frontier likelihood, which is finite and attained. On simulated panels this supremum is reached along a path that first dips below the interior local maximum and then rises past it, so an unconstrained search that escapes the interior basin returns σv≈0\sigma_v \approx 0 together with meaningless standard errors; when λ\lambda is large and TT small, no interior maximum need exist at all. tfe_lambda_max bounds the search accordingly, and a fit that pins at the bound warns. This is a property of the estimator, not of the implementation, and is one of the motivations for "TFE_WMLE".

Both problems are small-TT phenomena. Simulations in which TT grows with NN fixed at 100 (truth λ=3.333\lambda = 3.333) recover λ^=\hat\lambda = 88.5, 4.8, 3.6, 3.6, 3.5 at T=T = 10, 20, 30, 40, 50: the fit is pinned at the bound at T=10T = 10 and has converged by T=50T = 50. The degeneracy and the incidental-parameters bias dissolve together as TT grows, so tfe_lambda_max matters on short panels and is irrelevant on long ones. "TFE_WMLE", having no incidental parameters, is unaffected by TT either way.

Other likelihood-based panel models

"K1990" and "K1990modified" (Kumbhakar, 1990) extend the same error-components frontier with richer time paths for inefficiency: "K1990" uses Bit=(1+exp⁡(bt+ct2))−1B_{it} = (1 + \exp(b t + c t^2))^{-1} and "K1990modified" uses Bit=1+d(t−Ti)+e(t−Ti)2B_{it} = 1 + d(t - T_i) + e(t - T_i)^2, against "BC92"'s exp⁡(−η(t−Tref))\exp(-\eta(t - T_{ref})) and "PL80"'s time-invariant Bit=1B_{it} = 1. All four share one likelihood, differing only in BitB_{it}. Note that "K1990"'s b and c are weakly identified – quite different pairs produce near-identical fitted time paths – so the fitted BitB_{it} path is more interpretable than either coefficient on its own.

"PL80" (time-invariant panel inefficiency, Pitt and Lee, 1980, JoE) and "BC92" (time-varying decay, Battese and Coelli, 1992, JPA) are both natively estimated maximum-likelihood implementations of the same "error components frontier": y_it = x_it'beta + v_it - B_it*u_i (production; sign flips for cost), with v_it iid normal, u_i iid half-normal and constant across time within a firm, and B_it = 1 for "PL80" or B_it = exp(-eta*(t - Tref)) for "BC92" (Tref being the last time period in the whole panel). The closed-form log-likelihood is obtained by integrating u_i out of the joint density of (epsilon_i, u_i); it nests the ordinary cross-sectional normal-half-normal likelihood when every firm has a single observation. Verified against frontier::sfa() (matching coefficients, log-likelihood, and frontier::efficiencies()-style predicted efficiencies to numerical precision across balanced/unbalanced panels and production/cost specifications) before that dependency was removed. The time argument above supplies each observation's period for "BC92"'s decay term; if left NULL, a sequential within-individual time index is constructed from row order, which assumes data is already sorted chronologically within each individual.

Distribution-free estimators

"SSFE" is the classical Schmidt and Sickles (1984, JBES) fixed-effects (LSDV) estimator: a standard within/fixed-effects panel regression, with technical inefficiency read directly off the estimated firm effects (each firm's distance from the best-performing firm in the sample) rather than estimated by maximum likelihood. It requires no numerical optimization and returns no log-likelihood – logLik()/AIC()/BIC() are not defined for it (see sfareg-methods). Note this is a different model from "TFE"/"TFE_WMLE" above, despite the similar names: both of those are maximum-likelihood estimators that allow a separate stochastic noise term on top of the fixed effects, while "SSFE" attributes all within-firm deviation to inefficiency directly.

The Schmidt–Sickles family. "SSFE", "SSRE" and "SSCRE" are the within, GLS and correlated-random-effects members of one family: all three read inefficiency off a firm effect as distance from the best firm, and none assumes a distribution for uu. The choice among them is the standard panel trade-off, with a frontier-specific twist. Random effects is the more efficient estimator and is the only one of the three that identifies coefficients on time-invariant regressors, which the within transformation sweeps away – but it requires the effects to be uncorrelated with the regressors, and in a frontier model that assumption says a firm's inefficiency is unrelated to its input choices, which is close to saying managers do not respond to their own productivity.

"SSCRE" is the answer to that. Adding the within-firm means of the time-varying regressors (Mundlak 1978) models the correlation instead of assuming it away; the slopes on the original regressors then equal the within estimates exactly – not approximately – while time-invariant regressors remain identified. The coefficients on the added means are reported with a .mean_ prefix, and testing them jointly against zero is the Mundlak form of the Hausman test: if they are indistinguishable from zero, "SSRE" was safe. A regressor already constant within firm has a group mean identical to itself, so it contributes no Mundlak term; those are exactly the regressors random effects exists to identify, and they are left alone. The variance components, and so θ\theta and the BLUP-based inefficiencies, are the Swamy–Arora components of the model without the Mundlak means, balanced or unbalanced. On a balanced panel that is what plm's Swamy–Arora step returns; on an unbalanced one that step is singular, because in the between-firm dimension each mean duplicates its regressor, and "SSCRE" obtains the identical components through plm's Wallace–Hussain step on the augmented model, checking that they agree.

Time-varying fixed effects without a distribution for u. "CSS" and "LS" sit between "SSFE", which holds inefficiency fixed over time, and the maximum-likelihood panel models, which let it move but only by assuming a distribution for uu and a decay path for it. Both instead let the firm effect itself vary with time and read inefficiency off it, so neither assumes any distribution and neither returns a log-likelihood.

They differ in what they are willing to spend on flexibility. Cornwell, Schmidt and Sickles (1990) give every firm its own quadratic in time, αit=θi1+θi2t+θi3t2\alpha_{it} = \theta_{i1} + \theta_{i2}t + \theta_{i3}t^2: 3N3N parameters, firms may overtake one another, and Ti≥4T_i \ge 4 is required before a firm contributes to β\beta at all (a firm observed three times is fitted exactly by its own quadratic and its residuals are identically zero – sfa warns rather than letting such firms silently dilute the estimate). Lee and Schmidt (1993) impose one common temporal pattern scaled per firm, αit=δtαi\alpha_{it} = \delta_t\alpha_i: N+T−1N + T - 1 parameters, and the ranking of firms cannot change, only the spread. Cross-over versus no cross-over is the substantive choice between them, not a technical one.

"CSS" is fitted by residualizing yy and XX on each firm's own (1,t,t2)(1, t, t^2) rather than by building 3N3N dummy columns; a regressor spanned by those quadratics – a pure time trend, for instance – is a rank failure and is refused rather than reported as a small coefficient. "LS" is a rank-one factor model, so it is fitted by alternating least squares, which handles an unbalanced panel with no special-casing; δ1=1\delta_1 = 1 fixes the scale that δtαi\delta_t\alpha_i leaves free, with a fallback to a norm constraint when the first period's effect is near zero.

For both, inefficiency is identified only relative to the best firm in each period – uit=max⁡jαjt−αitu_{it} = \max_j \alpha_{jt} - \alpha_{it} – which is what makes them time-varying at all; comparing against a single overall best would collapse them back to a time-invariant ranking. Neither absorbs an intercept separately: it goes into the firm effect, as in "SSFE", so coef() returns slopes only.

Kneip, Sickles and Song (KSS)

Kneip, Sickles and Song (2012). "KSS" writes the firm effect as αit=∑r=1Lθirgr(t)\alpha_{it} = \sum_{r=1}^{L}\theta_{ir} g_r(t), where both the basis functions grg_r and the dimension LL are estimated from the data rather than assumed. That makes explicit what the other time-varying estimators here are special cases of:

model LL basis
"SSFE" 1 fixed, constant in tt
"LS" 1 free
"CSS" 3 fixed to {1,t,t2}\{1, t, t^2\}
"KSS" estimated estimated

Estimation follows the three steps of the original: cross-sectional centering by period (which imposes ∑iui(t)=0\sum_i u_i(t) = 0 and removes any common function of time), then smoothing of each firm's residual trajectory followed by an eigendecomposition of their empirical covariance to give g^r\hat g_r, then loadings by least squares and the effects rebuilt from them – iterated with β\beta until the residual sum of squares settles. Smoothing before the eigendecomposition is what makes the estimated basis a set of smooth functions rather than TT arbitrary period effects, and is the whole difference between this and an unrestricted factor model.

Two tuning choices are therefore unavoidable, and both are exposed. kss_smooth defaults to generalized cross-validation. kss_L defaults to the Bai and Ng (2002) ICp2IC_{p2} criterion – not the threshold rule of the original paper, which is stated here because it matters: ICp2IC_{p2} is the standard criterion for this problem and the literature that motivates the estimator treats Bai and Ng (2002) and Bai (2009) as the comparable methods, but a different rule can select a different LL. Set kss_L explicitly to take the choice out of the criterion's hands, and inspect $kss$eigenvalues to see whether the choice was close.

The criterion needs a long enough panel, and says so. ICp2IC_{p2} works because the residual variance flattens out once the real factors are in and the penalty then dominates. On a short panel that never happens: if LL is allowed up to T−1T-1 the factors can span nearly the whole time dimension, the variance collapses toward zero, and no penalty competes. Measured at n=100n = 100 over five seeds and two true ranks, an uncapped criterion returned its maximum on all ten designs at T=6T = 6 and again at T=8T = 8. The automatic search is therefore capped at ⌊T/2⌋\lfloor T/2 \rfloor, which recovers the true rank exactly from T=8T = 8 upward; at T=6T = 6 it is right for a one-factor structure and occasionally one too high for a two-factor one. Whenever the automatic choice lands on that cap psfm warns, because a selection at the boundary is not a free choice – read it as "at least this many", inspect the eigenvalues, and prefer a longer panel to a larger kss_L_max.

One consequence of the period centering is worth knowing before comparing models: "KSS"'s alpha_hat carries no period mean, while "CSS"'s and "LS"'s do. The two therefore differ by a per-period constant by construction and should not be correlated against each other directly. u_hat is a within-period contrast and is invariant to that shift, so it is the quantity to compare across these models – and is what the estimators are for.

Mundlak adjustment terms (mundlak)

Mundlak adjustment terms (mundlak). The true random effects model assumes the firm effect is independent of the regressors. In production data it usually is not – better managers choose different input levels – and the effect is then split between the slopes and inefficiency, biasing the technology parameters and understating inefficiency. Mundlak's (1978) device models the correlation instead of assuming it away, αi=π′xˉi+δi\alpha_i = \pi'\bar{x}_i + \delta_i. Substituting into the frontier leaves the same likelihood with a wider design matrix, which is why this is an argument rather than a new model_name.

Karagiannis and Kellermann (2019) widen it to αi=π′xˉi∗+γ′zi+δi\alpha_i = \pi'\bar{x}^*_i + \gamma'z_i + \delta_i, adding environmental variables – conditions outside the producer's control, such as agro-ecological conditions or the geography an airline serves. Those enter the formula as ordinary time-invariant regressors; mundlak supplies the piece a user cannot easily build by hand, the within-firm means computed against the same panel index the fit uses. Naming a subset of the regressors gives their Hausman-Taylor variant, in which only some regressors are taken to correlate with the effect.

Measured over 40 replications with N=80N = 80, T=6T = 6, on a design where the firm effect is correlated with xˉ\bar{x} by construction, and on a control where it is not:

design estimator bias RMSE
effect correlated plain +0.0479 0.0570
effect correlated mundlak -0.0061 0.0310
no correlation plain -0.0033 0.0253
no correlation mundlak -0.0068 0.0311

So it removes the bias where there is one and nearly halves RMSE – and where there is no correlation it costs about 23% in RMSE for nothing. It is not free, and it is not a default.

A variable that is already constant within firm gets no mean column: its own mean is itself, so the two would be exactly collinear. Every mundlak fit also emits a collinearity warning from the starting-value regression, which is expected and harmless – in a between-individual regression xx and xˉ\bar{x} are the same column by construction. The starting values fall back to pooled OLS and the likelihood is estimated with the full requested formula.

Profiling a fitted panel likelihood

Profiling a fitted panel likelihood. With keep_objective = TRUE the fitted object carries $objective, the same negative-summed-log-likelihood the optimizer minimised. Evaluating it away from the optimum is the only way to ask whether a parameter is weakly identified, as distinct from badly estimated:

fit <- psfm(y ~ x1 + x2, "GTRE", d, individual = "id",
            estimator = "sml", keep_objective = TRUE)
ih   <- which(names(coef(fit)) == "sigh")
grid <- seq(0.05, 1, by = 0.05)
ll   <- vapply(grid, function(v) {
          p <- fit$opt$par; p[ih] <- v; -fit$objective(p) }, numeric(1))
plot(grid, ll - max(ll), type = "l")

A flat curve means no estimator could recover the parameter from these data; a sharp one means the difficulty lies elsewhere. For GTRE's sigh the curve spans over 200 log units at N=200N = 200 and sharpens with NN, which is what ruled out weak identification as the explanation for its convergence behaviour.

Optimizer defaults

Note on default iteration counts: psfm()'s defaults were raised in version 1.1.3, from 100/10/10 for maxit.bobyqa/maxit.psoptim/maxit.optim to 5000/100/1000, and maxit.nlminb (previously hard-coded) is now an argument. The old values were binding rather than merely economical: the K1990/K1990modified branch already carried a note that 100 bobyqa evaluations left its seven-parameter fits several log-likelihood units short of the optimum purely on the iteration cap, and GTRE_FML at N = 500, T = 10 roughly halves its root-mean-square error against known true values when the caps are lifted, at about 1.5 times the run time. They remain below sfm()'s defaults because several psfm() models (GTRE, GTRE_Z, TRE, TRE_Z) are estimated by simulated maximum likelihood over Halton draws, so each objective evaluation is far more expensive than the closed-form likelihoods used elsewhere. Lower them explicitly if a fast approximate call is wanted.

Value

An object of class "sfareg" containing components that vary by model. All models return:

out

A matrix with parameter estimates, standard errors, and t-values.

opt

A list containing the optimization results from the final optimization procedure (not returned for GTRE_SEQ1 and GTRE_SEQ2).

total_time

The total computation time for model estimation.

start_v

The starting values used in the optimization (not returned for GTRE_SEQ1 and GTRE_SEQ2).

model_name

The name of the panel stochastic frontier model estimated.

formula

The formula used in the model specification.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE).

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

call

The matched call.

data

The data used in estimation.

Additional model-specific components:

For GTRE_FML model:

start_search

A list recording the starting-value search: n_tried, the log-likelihood at each candidate (loglik), and which was chosen ("random-effects" or "two-step"). NULL when start_val was supplied.

U

Predicted transient (time-varying) technical efficiency for each observation.

H

Predicted persistent (time-invariant) technical efficiency for each individual, named by individual.

For GTRE and GTRE_Z models:

H

Predicted time-invariant technical efficiency for each individual.

For GTRE_Z model:

out_opg

OPG (outer product of gradients) standard errors and t-values along with parameters.

out_sandwich

sandwich standard errors and t-values along with parameters.

For GTRE, GTRE_Z, TRE and TRE_Z models:

U

Predicted time-varying technical efficiency for each observation.

For TFE and TFE_WMLE models:

r_hat_m

Estimated individual-specific fixed effects, named by individual. For "TFE" these are the maximum-likelihood estimates of αi\alpha_i directly; for "TFE_WMLE" they are recovered after estimation from the mean residual plus 2/π σu\sqrt{2/\pi}\,\sigma_u, since the within likelihood does not identify them.

exp_u_hat

Predicted technical efficiency, E[exp⁡(−u)∣ϵ]E[\exp(-u)|\epsilon] (Battese and Coelli, 1988).

For TFE model only:

u_hat

Predicted inefficiency, E[u∣ϵ]E[u|\epsilon] (Jondrow et al., 1982).

For FD model:

u_hat

Predicted technical efficiency in levels.

h_hat

Estimated z heterogeneity function values.

exp_u_hat

Predicted technical efficiency.

For GTRE_SEQ1 and GTRE_SEQ2 models:

other_parms

A matrix of additional parameters (lambda, sigma, beta_0 for SEQ1; sigma_u, sigma_v, sigma_h, sigma_r, lambda, sigma for SEQ2).

For SSFE model:

alpha_hat

The estimated firm-specific fixed effects (named by individual).

u_hat

Estimated technical inefficiency for each firm, as its distance from the best-performing firm's fixed effect.

exp_u_hat

Predicted technical efficiency, exp(-u_hat).

For the GTRE model:

sigh_at_bound, sigr_at_bound

TRUE when that persistent scale has collapsed to the zero boundary, judged relative to sigma. Frequently the CORRECT maximum likelihood estimate rather than a failure – see ‘Details’ – so it is reported rather than suppressed, and psfm warns naming the other persistent scale, since the two must be read together.

For CSS and LS models:

alpha_hat

The fitted firm effect for every observation, αit\alpha_{it} – time-varying, unlike SSFE's one value per firm.

u_hat, exp_u_hat

Inefficiency and efficiency per observation, measured against the best firm in the same period.

frontier_alpha

The best firm effect in each period, i.e. the benchmark u_hat is measured from.

sigma_v, residuals, df.residual

The residual standard deviation, the residuals net of the firm effect, and the residual degrees of freedom (NT−K−3NNT - K - 3N for CSS, NT−K−(N+T−1)NT - K - (N + T - 1) for LS).

theta

CSS only. An N×3N \times 3 matrix of the firm-specific quadratic coefficients.

delta, alpha_i, ls_iterations, ls_converged

LS only. The common temporal pattern (with delta[1] == 1), the firm-specific scales, and the state of the alternating least-squares iteration.

For SSRE and SSCRE models:

alpha_hat

The BLUPs of the firm effects, one per firm. Under SSCRE these are net of the part the Mundlak means explain.

theta, ercomp

The GLS transformation parameter and the full variance-components decomposition from ercomp.

mundlak_terms

SSCRE only. The names of the added group-mean regressors.

plm_fit

The underlying plm object, for users who want its own methods.

For the KSS model:

kss

A list: the selected dimension L, the smoothing penalty kappa, the estimated basis (a T×LT \times L matrix of orthonormal columns), the n×Ln \times L matrix of loadings, the full eigenvalues sequence the dimension was chosen from, and the iteration state.

For PL80 and BC92 models:

exp_u_hat

Predicted technical efficiency for each observation, E[exp(-B_it*u_i)|epsilon_i] (generalizes Battese and Coelli's (1988, JoE) point estimator to the panel case, pooling all of a firm's observations to predict u_i and scaling by B_it per period).

Note

Standard errors require optHessian = TRUE.

The GTRE_SEQ1 and GTRE_SEQ2 models use sequential estimation methods and do not return optimization objects or starting values. The SSFE, CSS, LS, SSRE, SSCRE and KSS models are likewise not maximum-likelihood based and return no optimization object, starting values, or log-likelihood. All panel models require the individual argument to identify panel units.

Author(s)

David H. Bernstein

References

Fillipini and Greene (2016, JPA); Colombi, Kumbhakar, Martini and Vittadini (2014, JPA) on the closed-skew-normal four-component likelihood, with Colombi (2010) and Colombi, Martini and Vittadini (2011) on the two-step estimator used to seed it; Greene (2005, JoE and JPA); Wang and Ho (2010, JoE); Chen, Schmidt and Wang (2014, JoE); Kumbhakar (1990, JoE); Pitt and Lee (1980, JoE); Battese and Coelli (1992, JPA); Battese and Coelli (1988, JoE); Jondrow, Lovell, Materov and Schmidt (1982, JoE); Schmidt and Sickles (1984, JBES)

See Also

sfm for cross-sectional models, psfm_bootstrap for bootstrap standard errors, data_gen_p for simulating panel data with known true parameters, and sfareg-methods for extracting results.

Examples

library(sfa)     

## A deliberately small panel with few simulation draws, so the examples
## build quickly. Do not read the estimates as a serious fit. TRE_Z becomes
## unstable below about 60 firms, so this is close to the floor.
data_trial <- data_gen_p(t=6, N=60, rand = 100,
                         sig_u = 1,  sig_v = 0.3,
                         sig_r = .2, sig_h = .4,
                         cons = 0.5, beta1 = 0.5,
                         beta2 = 0.5)

## rand.psoptim is not optional with PSopt = TRUE: the particle swarm draws
## from the session RNG, so results otherwise change between builds.
max_tre_z   <-  psfm(formula    = y_tre_z ~ x1 +x2| z_gtre,
                     model_name = "TRE_Z",
                     data       = data_trial,
                     individual = "name",
                     halton_num = 50,
                     rand.psoptim = 7L,
                     PSopt      = TRUE)

## Time-varying fixed effects, no distribution assumed for u. CSS gives each
## firm its own quadratic in time; LS gives all firms one common pattern.
css_fit     <-  psfm(formula = y_tre ~ x1 + x2, model_name = "CSS",
                     data = data_trial, individual = "name", time = "year")
ls_fit      <-  psfm(formula = y_tre ~ x1 + x2, model_name = "LS",
                     data = data_trial, individual = "name", time = "year")
ls_fit$delta      # the common temporal pattern, normalized to delta[1] = 1

## The Schmidt-Sickles family: within, GLS, and correlated random effects.
## SSCRE's slopes equal SSFE's within slopes exactly (Mundlak 1978); the
## .mean_ coefficients are the Mundlak form of the Hausman test.
ssre_fit    <-  psfm(formula = y_tre ~ x1 + x2, model_name = "SSRE",
                     data = data_trial, individual = "name", time = "year")
sscre_fit   <-  psfm(formula = y_tre ~ x1 + x2, model_name = "SSCRE",
                     data = data_trial, individual = "name", time = "year")

## Kneip, Sickles and Song (2012): the temporal basis and its dimension are
## both estimated. Inspect what the criterion chose before trusting it.
kss_fit     <-  psfm(formula = y_tre ~ x1 + x2, model_name = "KSS",
                     data = data_trial, individual = "name", time = "year")
kss_fit$kss$L                 # selected dimension
kss_fit$kss$eigenvalues       # was the choice close?

Parametric Bootstrap for psfm() Panel Stochastic Frontier Models

Description

Parametric bootstrap standard errors for a panel frontier fitted by psfm. For each of BOOT replications a new response is simulated from the fitted parameters, with a data-generating process specific to the model, and the model is refitted in parallel. Bootstrap standard errors and t-values are returned for every parameter in psfm_object$out.

Supported models: "GTRE_Z", "TRE_Z", "GTRE", "GTRE_FML", "TRE", "TFE", "TFE_WMLE" and "FD". See ‘Supported models’ in Details for why the others are refused.

Usage

psfm_bootstrap(psfm_object,
               numCores,
               BOOT,
               individual,
               h_type        = c("auto", "none", "scalar", "parametric"),
               maxit.psoptim = 1000,
               seed_offset   = 0,
               write_back    = TRUE,
               pkgs          = c("sfa", "Formula", "truncnorm"),
               inefdec,
               rand.gtre     = NULL,
               rand.psoptim  = NULL,
               maxit.bobyqa  = 1,
               maxit.optim   = 1)

Arguments

psfm_object

A fitted model object of class "sfareg" returned by psfm, with model_name %in% c("GTRE_Z", "TRE_Z", "GTRE", "GTRE_FML", "TRE", "TFE", "TFE_WMLE", "FD"). Note that psfm(model_name = "GTRE") returns model_name = "GTRE_FML" under its default estimator = "fiml", which is why that value is supported here rather than only "GTRE". Must contain $out, $data, $formula, and $model_name; $H is additionally required for "GTRE"/"GTRE_FML"/"GTRE_Z" (the only models that return it) and $r_hat_m is additionally required for "TFE" and "TFE_WMLE".

numCores

Integer. Number of cores for the parallel cluster created internally via makeCluster.

BOOT

Integer. Number of bootstrap replications (e.g. 199, 399, 999).

individual

Character. Name of the individual/panel-ID column in psfm_object$data (e.g. "firm.id").

h_type

One of "auto", "none", "scalar", or "parametric". Only meaningful for model_name %in% c("GTRE_Z", "TRE_Z", "GTRE", "GTRE_FML", "TRE") (ignored for "TFE"/"TFE_WMLE"/"FD", which have no random persistent-inefficiency h term); controls how the individual-specific, time-invariant inefficiency term h is simulated:

"none"

The model has no h term at all (TRE, TRE_Z).

"scalar"

h_i ~ abs(N(0, sigma_h)), a single scalar standard deviation – used automatically for bare "GTRE" (which has no h-covariates at all, only a single fitted sigh).

"parametric"

h_i ~ abs(N(0, exp(data_h %*% h_vec))), using the third right-hand-side part of the model formula as the h-covariates – used automatically for "GTRE_Z".

"auto"

(Default) Resolves per model_name: "parametric" for "GTRE_Z", "scalar" for bare "GTRE" and "GTRE_FML", "none" for "TRE_Z"/"TRE".

maxit.psoptim

Integer. Passed through to psfm() as maxit.psoptim on each bootstrap re-estimation. Default 1000.

seed_offset

Whole number selecting a block of seeds: replication b is seeded with set.seed(b + seed_offset * 100000), so runs with different seed_offset never share a seed (for BOOT up to 100000) and can be pooled as independent. Default 0, which seeds replication b with b, as before. Through 1.2.0 the seed was b + seed_offset, so consecutive offsets repeated all but one replication of each other.

write_back

Logical. If TRUE (default), the returned list includes model, a copy of psfm_object with the bootstrap standard errors and t-values written into $out[,2] and $out[,3]. If FALSE, only the raw bootstrap results are returned.

pkgs

Character vector of package names to load on each cluster worker via clusterCall. Default c("sfa", "Formula", "truncnorm") – truncnorm is only actually used when bootstrapping an "FD" fit, but is included in the default so a single default works for every supported model.

inefdec

Logical: TRUE for a production frontier, FALSE for cost. Sets the sign of the simulated inefficiency and is passed through to psfm() on each bootstrap re-estimation. Leave it out: it is recovered from psfm_object$call, so it matches the fit. Supplying a value that differs from the fit's resamples from a frontier of the opposite orientation, and nothing downstream reports that. It is needed only when the value cannot be recovered, for example when the fit passed a variable that no longer exists, and then the function says so.

rand.gtre

Passed through to psfm() as rand.gtre on each bootstrap re-estimation. Default NULL.

rand.psoptim

Passed through to psfm() as rand.psoptim on each bootstrap re-estimation. Default NULL.

maxit.bobyqa

Integer. Passed through to psfm() as maxit.bobyqa on each bootstrap re-estimation. Default 1.

maxit.optim

Integer. Passed through to psfm() as maxit.optim on each bootstrap re-estimation. Default 1.

Details

Supported models

Any other model_name is refused with an error. The moment-based, sequential and LSDV estimators ("GTRE_SEQ1", "GTRE_SEQ2", "SSFE", "SSRE", "SSCRE", "CSS", "LS", "KSS") are not maximum likelihood, so there is no fitted likelihood to resample from in the same sense. The time-pattern models ("PL80", "PL80_MVTN", "BC92", "K1990", "K1990modified") do not return the $U/$H structure this function relies on.

How each model is simulated

The supported models fall into three families, each with its own data-generating process and $out row layout. The header comment of R/psfm_bootstrap.R gives the full derivation, including how each was cross-checked against that model's own likelihood in psfm.R.

"randeff" family (GTRE_Z, TRE_Z, GTRE, TRE)

y* = X %*% beta_hat + v + u + r + h (sign per inefdec), with v ~ N(0, sigma_v^2), u half-normal (covariate-driven sigma for GTRE_Z/TRE_Z, a single homoskedastic scalar for bare GTRE/TRE, which have no z pipe at all), r ~ N(0, sigma_r^2) (one draw per individual, repeated over time), and h per h_type (GTRE/GTRE_Z only). $out row layout: sigv, sigr, [x-block], [z-block], [h-block] for GTRE_Z/TRE_Z; lambda, sigma, sigr, [sigh], [x-block] for bare GTRE/TRE (note the different reparameterization – see psfm's own model-family sigma-link inconsistency note).

"tfe" family (TFE, TFE_WMLE)

Fixed-effects model: the individual effects are held FIXED at psfm_object$r_hat_m's original point estimates across every replication (not redrawn – redrawing them would contradict treating them as fixed rather than random), and only v ~ N(0, sigma_v^2) and u ~ |N(0, sigma_u^2)| (both derived from the fitted lambda/sig, or gamma/sig if the original fit used psfm(..., gamma = TRUE)) are redrawn. $out row layout: lambda (or gamma), sig, [x-block] (no intercept row – the within-transformation removes it).

"fd" family (FD)

Wang and Ho (2010) first-difference estimator: a single time-invariant u_i ~ TruncatedNormal(mean = mu, sd = sqrt(sig_u2), lower = 0) draw per individual (via rtruncnorm), scaled by a deterministic time-varying factor h_it = exp(z_it %*% delta), plus v_it ~ N(0, sig_v2). $out row layout: sig_u2, sig_v2, mu, [x-block], [z-block] (no intercept row in either the x- or z-block).

Checks before the first replication

psfm_bootstrap validates that nrow(psfm_object$out) matches the expected layout for the resolved model/h_type before running any bootstrap replications, and stops with an informative error if it does not. For GTRE/GTRE_Z it similarly validates up front that length(psfm_object$H) equals the number of unique individuals implied by the individual column ($H is one value per individual, not one per observation) – TRE/TRE_Z/TFE/TFE_WMLE/FD do not return $H at all, so boot_eff_h is simply NULL for those five models (not an error).

Refitting, and replications that fail

The model is re-estimated on each simulated dataset via sfa::psfm(), executed in parallel across replications using parLapply (or pblapply for a live progress bar, when the pbapply package is installed). Whether the psoptim (particle-swarm) optimizer stage is used on each refit is model-dependent: it is enabled for the "randeff" family (matching this function's original behavior, tested clean), but disabled for "TFE" and "TFE_WMLE" – testing found psoptim's wide exploration made TFE repeatedly collapse onto a degenerate variance-boundary solution (reproduced even refitting the SAME real data across different psoptim seeds), a problem that disappeared entirely once that stage was skipped (matching psfm()'s own default of PSopt = FALSE). As a defense-in-depth safety net across every family, any replication whose refit still lands on a degenerate variance-boundary mode (a scale parameter pinned near its numerical floor – the same well-understood boundary-mode phenomenon documented for ttsfm()'s TTNE/TTHN models) is treated as a per-replication FAILURE rather than silently corrupting the bootstrap standard error with a meaningless draw.

Re-estimation failures (optimizer errors, or the degenerate-mode check just described) are caught per replication, recorded with their message, and set to NA in the output matrices rather than stopping the bootstrap; a single warning at the end lists every failed replication index together with its message. Bootstrap standard errors are computed with na.rm = TRUE, so they remain well-defined even when some replications failed.

Requirements

Requires the parallel and Formula packages (and truncnorm when bootstrapping an "FD" fit), and requires sfa (and any other packages named in pkgs) to be installed and loadable on each cluster worker.

Value

A list with components:

boot_par

A BOOT x (nrow(psfm_object$out) + 2) matrix of bootstrapped parameter draws, with the final two columns "loglik" and "hours". Rows corresponding to failed replications are NA.

boot_eff

A BOOT x nrow(psfm_object$data) matrix of bootstrapped per-observation efficiency-score draws ($U for the "randeff" family, $exp_u_hat for "TFE"/"TFE_WMLE"/"FD" – the field is named differently across these models in psfm.R, but always collected into this same boot_eff matrix here). Rows corresponding to failed replications are NA.

boot_eff_h

Only non-NULL for model_name %in% c("GTRE", "GTRE_FML", "GTRE_Z") (the only models that return $H at all); NULL otherwise. A BOOT x length(psfm_object$H) matrix of bootstrapped time-invariant, individual-specific inefficiency-score ($H) draws – one column per unique individual (not one column per observation; $H does not vary over time within an individual). Columns are named by the individual ID values. Rows corresponding to failed replications are NA.

se

Named numeric vector of bootstrap standard errors for every parameter in psfm_object$out.

tval

Named numeric vector of bootstrap t-values (estimate / boot SE) for every parameter in psfm_object$out.

failures

Integer vector of replication indices b for which re-estimation failed, or NULL if none failed.

model

Only present when write_back = TRUE. A copy of psfm_object with $out[,2] and $out[,3] replaced by the bootstrap standard errors and t-values.

Author(s)

David H. Bernstein and Christopher F. Parmeter

See Also

psfm, parLapply, pblapply, Formula

Examples

## Not run: 
## Not run: a meaningful bootstrap needs BOOT in the hundreds, each
## replication refitting the model on a PSOCK cluster, so this takes
## minutes rather than seconds.

data_trial <- data_gen_p(t = 5, N = 50, rand = 100,
                         sig_u = 1,   sig_v = 0.3,
                         sig_r = 0.2, sig_h = 0.4,
                         cons  = 0.5, beta1 = 0.5, beta2 = 0.5)

GTRE_ZZ <- psfm(y_gtre_zz ~ x1 + x2 | z_gtre | zp_gtre,
                model_name = "GTRE_Z",
                data       = data_trial,
                individual = "name")

boot_results <- psfm_bootstrap(
  psfm_object = GTRE_ZZ,
  numCores    = 2,
  BOOT        = 199,
  individual  = "name",
  h_type      = "auto"
)
## inefdec is left out on purpose: it is recovered from the fit, which used
## psfm()'s default inefdec = TRUE because y_gtre_zz is a production frontier.

## Bootstrap standard errors / t-values for every model parameter
boot_results$se
boot_results$tval

## Bootstrap draws of time-varying (U) and time-invariant (H)
## efficiency/inefficiency scores, one row per replication
boot_results$boot_eff
boot_results$boot_eff_h

## Fitted object with bootstrap SEs/t-values written into $out
boot_results$model$out

## End(Not run)

Stochastic Frontier Models with Sample Selection or Technology Choice

Description

Fits one of two models in which the frontier and a binary choice are estimated together. model_name picks the model:

model_name model what drives the choice
"greene" (default) Greene (2010): one frontier, seen only for units that select into the sample the noise vv
"kts" Kumbhakar, Tsionas and Sipilainen (2009): a frontier and two scales per technology inefficiency uu

The two are different models, not variants of one, and neither nests the other. "greene" is a two-step estimator: a probit on the full sample, then maximum simulated likelihood for the frontier on the selected rows. "kts" is single-step maximum likelihood over the whole sample, with the integral over uu done by Gauss–Legendre quadrature.

Usage

selsfm(selection, frontier, data, model_name = c("greene", "kts"),
       n_nodes = 64, Nsim = "auto",
       sim_type = c("halton", "sobol", "torus", "uniform"),
       antithetics = FALSE, seed = NULL, inefdec = TRUE,
       maxit.bobyqa = 10000, maxit.psoptim = 1000, maxit.optim = 1000,
       start_val = FALSE, PSopt = FALSE, optHessian = TRUE,
       Method = "L-BFGS-B", verbose = FALSE, keep_objective = FALSE,
       rand.psoptim = NULL)

Arguments

selection

A two-sided formula for the binary equation, d ~ z1 + z2: selection into the sample for "greene", the technology indicator for "kts". For "greene" the response may be 0/1, logical, or a two-level factor, and the probit uses all rows of data. For "kts" it must be 0/1.

frontier

A two-sided formula for the frontier, y ~ x1 + x2. "greene" fits it on the selected rows only, so y may be NA where d == 0. "kts" fits a separate copy for each technology and uses every row; a row with any missing value is dropped.

model_name

"greene" (default) or "kts"; see ‘Description’ and ‘Details’.

n_nodes

"kts" only. Gauss–Legendre nodes for the integral over uu. The half-normal integral is mapped to [0,1][0,1] by its own CDF, so no truncation point is needed.

data

A data.frame containing both equations' variables, with the unselected rows present. Unlike the rest of the package neither formula takes a | segment – the two equations are separate arguments – and a pipe is an error rather than being silently ignored.

Nsim

"greene" only, like sim_type, antithetics and seed. Number of simulation draws per selected observation. "auto" (the default) uses max(200, ceiling(3*sqrt(n_selected))). Simulated ML is consistent only if the draw count grows with the sample size; a numeric value below the automatic floor warns.

sim_type

Low-discrepancy sequence for the draws, passed to the same generator the other simulated-ML models use.

antithetics

Mirror the draws to halve the number of independent points.

seed

Optional seed for the draws. The RNG state is saved and restored, so a fit does not disturb the caller's stream.

inefdec

TRUE (default) for a production frontier, where inefficiency is subtracted; FALSE for a cost frontier.

maxit.bobyqa, maxit.psoptim, maxit.optim

Iteration caps for the three optimizer stages.

start_val

For "greene", TRUE names the starting-value vector returned as $start_v; starting values cannot be supplied. For "kts", a numeric vector is used as the starting values, in the order of the "kts" parameters listed under ‘Value’ but with the four scales given as logs.

PSopt

Run the particle-swarm stage between BOBYQA and optim.

optHessian

Compute the Hessian, and with it the standard errors.

Method

Method passed to optim for the final stage. Must be "L-BFGS-B": it is the only optim method that applies the lower and upper bounds this package uses to keep scale parameters positive, and any other value is refused with an error rather than quietly optimized without them.

verbose

Report optimizer progress.

keep_objective

Logical. Retain the second-stage likelihood closure on the fitted object as $objective, so the score-based tools can difference it: vcov(type = "bhhh") (OPG/BHHH standard errors), influence_sfa, the sandwich methods, TIC and vuong. New in 1.2.1. The contributions are over the selected observations, which is the sum the second stage maximises. This is a two-step estimator: α^\hat\alpha comes from the first-stage probit and enters the second stage as if known, so the OPG ignores first-stage estimation error exactly as the stored Hessian already does. It is the same conditional-on-α^\hat\alpha variance computed a different way, not a correction for it. FALSE by default, because a closure carries its enclosing environment. "kts" ignores this argument and always returns $objective.

rand.psoptim

Optional seed for the particle-swarm stage.

Details

Greene (2010): selection through the noise

The model is Greene's Equation (10):

di=1[α′zi+wi>0],wi∼N(0,1)d_i = 1[\alpha'z_i + w_i > 0], \quad w_i \sim N(0,1)

yi=β′xi+vi−ui,observed only when di=1y_i = \beta'x_i + v_i - u_i, \quad \mathrm{observed\ only\ when\ } d_i = 1

ui=σU∣Ui∣,Ui∼N(0,1),vi=σVVi,Vi∼N(0,1)u_i = \sigma_U|U_i|, \quad U_i \sim N(0,1), \qquad v_i = \sigma_V V_i, \quad V_i \sim N(0,1)

(wi,vi)∼N2[(0,0),(1,ρσV;ρσV,σV2)](w_i, v_i) \sim N_2[(0,0), (1, \rho\sigma_V; \rho\sigma_V, \sigma_V^2)]

Selection operates through the noise vv, not through the inefficiency uu. That is the substantive assumption separating this model from Kumbhakar, Tsionas and Sipilainen (2009), where the selection disturbance is correlated with uu instead; the two answer different questions and are not nested.

Because uu cannot be integrated out in closed form, the log-likelihood is simulated (Greene's Equation (15)): for each selected observation RR draws of ∣Uir∣|U_{ir}| are taken and

log⁡L=∑di=1log⁡1R∑r=1Rϕ ⁣(virσV)1σVΦ ⁣(ρvir/σV+α′zi1−ρ2)\log L = \sum_{d_i=1} \log \frac{1}{R} \sum_{r=1}^{R} \phi\!\left(\frac{v_{ir}}{\sigma_V}\right)\frac{1}{\sigma_V} \Phi\!\left(\frac{\rho v_{ir}/\sigma_V + \alpha'z_i}{\sqrt{1-\rho^2}}\right)

with vir=yi−β′xi+σU∣Uir∣v_{ir} = y_i - \beta'x_i + \sigma_U|U_{ir}|. Everything is evaluated in logs and accumulated by log-sum-exp, since the summand underflows in the tails well before the total does.

A note on the published formula. Equations (11) and (13)–(15) of Greene (2010) write σε\sigma_\varepsilon in the denominator of the Φ\Phi argument. It must be σV\sigma_V: the term being standardized is vir=yi−β′xi+σU∣Uir∣v_{ir} = y_i - \beta'x_i + \sigma_U|U_{ir}|, which is the noise, and the conditional distribution of ww given vv for a bivariate normal with Var(w)=1\mathrm{Var}(w) = 1 and correlation ρ\rho is N(ρv/σV, 1−ρ2)N(\rho v/\sigma_V,\, 1-\rho^2). This implementation uses σV\sigma_V.

Standard errors are conditional on the first stage. Following the paper's two-step scheme, α\alpha is taken as given in the frontier likelihood. The reported standard errors come from the second-stage Hessian and therefore do not include the sampling variability of α^\hat\alpha; Greene applies the Murphy–Topel correction at this point, which is not implemented here. Treat them as a lower bound, and bootstrap over both stages when the precision matters.

Kumbhakar, Tsionas and Sipilainen (2009): choice through inefficiency

For technology Ii∈{0,1}I_i \in \{0,1\},

yi∣Ii,ui∼N(xi′βIi−ui, σv,Ii2),P(Ii=1∣ui)=Φ(zi′γ+δui),ui∣Ii∼N+(0,σu,Ii2).y_i \mid I_i, u_i \sim N(x_i'\beta_{I_i} - u_i,\ \sigma_{v,I_i}^2), \qquad P(I_i = 1 \mid u_i) = \Phi(z_i'\gamma + \delta u_i), \qquad u_i \mid I_i \sim N^+(0, \sigma_{u,I_i}^2).

δ\delta is the parameter the model exists for: it is the dual role of inefficiency, and δ>0\delta > 0 means the less efficient are more likely to choose technology 1. Neither two-step order works – the choice equation cannot be a probit because uu is unobserved, and fitting the frontiers first ignores the endogeneity of the choice – which is why the paper estimates in one step.

Because uu's distribution depends on II while II's probability depends on uu, the marginal density of uu is a mixture of the two regimes' half-normals, with weight ϕi\phi_i solving the paper's equation (9). The observation's density is then the same mixture of two one-dimensional integrals, its equation (14).

Recovery. On data simulated from this exact model with δ=0.9\delta = 0.9, the frontier coefficients, the four scales and δ\delta all recover, and improve with the sample:

parameter n = 1500 n = 6000 truth
δ\delta 1.361 0.922 0.900
σu,0\sigma_{u,0} 0.837 0.843 0.800
σu,1\sigma_{u,1} 0.632 0.640 0.600
choice intercept -0.511 -0.239 -0.200

δ\delta is the parameter that needs the most data, as one would expect of a coefficient identified only through the coupling between the two equations, but it is properly identified: holding the rest at the truth, the mean log-likelihood falls monotonically away from δ=0.9\delta = 0.9 in both directions, by about 0.069 per observation across [0.1,1.8][0.1, 1.8].

Value

An object of class "sfareg". As elsewhere in the package out is a p x 3 matrix, one row per parameter, with columns par, st_err and t-val. Both models return jlms, efficiency (exp(-jlms)), selection_formula and nobs.

For "greene" (model_name "SEL" on the fit), the parameters are sigma_u, sigma_v, rho, then the frontier coefficients, and nobs is the number of selected rows, the sample the second-stage likelihood sums over. Also returned:

alpha

First-stage probit coefficients.

probit

The fitted first-stage object.

jlms

E[ui∣εi]E[u_i \mid \varepsilon_i] by the simulated Bayes rule of Greene's Equation (21), computed over the same draws the likelihood used.

n_selected, n_total

Selected and total observation counts.

Nsim, sim_type

The draw count and sequence actually used.

For "kts" (model_name "KTS" on the fit), the parameters are the technology-0 frontier (t0.), the technology-1 frontier (t1.), sigma_v0, sigma_v1, sigma_u0, sigma_u1, the choice equation (choice.), then delta. nobs counts every row used, n_selected the rows with technology 1, and n_nodes the quadrature nodes.

References

Kumbhakar, S.C., Tsionas, E.G. and Sipilainen, T. (2009). Joint estimation of technology choice and technical efficiency: an application to organic and conventional dairy farming. Journal of Productivity Analysis, 31(3), 151–161.

Greene, W. (2010). A stochastic frontier model with correction for sample selection. Journal of Productivity Analysis, 34(1), 15–24.

Heckman, J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.

See Also

sfm for the frontier without a selection correction.

Examples

set.seed(1)
n  <- 800
z1 <- rnorm(n); x1 <- rnorm(n)
w  <- rnorm(n)
d  <- as.numeric(0.2 + 0.8 * z1 + w > 0)
v  <- 0.5 * (0.6 * w + sqrt(1 - 0.6^2) * rnorm(n))
y  <- 0.5 + 0.8 * x1 + v - abs(rnorm(n))
y[d == 0] <- NA
dat <- data.frame(y = y, x1 = x1, z1 = z1, d = d)

fit <- selsfm(selection = d ~ z1, frontier = y ~ x1, data = dat, Nsim = 100)
fit$out

Optimizer Diagnostics for Stochastic Frontier Fits

Description

Reports how the optimizer actually finished and how well the likelihood identifies the parameters: the convergence code and message, the eigenvalue spectrum and conditioning of the Hessian, the implied parameter correlations, and – where the objective was retained – the gradient at the reported optimum and a likelihood slice through each parameter.

Usage

sfa_diagnostics(object, ...)

## S3 method for class 'sfareg'
plot(x, which = 1:4, n_grid = 41, span = 0.25, ...)

## S3 method for class 'sfadiag'
print(x, ...)

Arguments

object, x

An object of class "sfareg" (for sfa_diagnostics and plot), or of class "sfadiag" (for print).

which

Which panels to draw: 1 the Hessian eigenvalue spectrum, 2 the parameter correlation matrix, 3 a likelihood slice per parameter, 4 the gradient. Panels that cannot be drawn are dropped silently; asking for only unavailable panels is an error rather than an empty plot.

n_grid

Number of points in each likelihood slice. Defaults to 41.

span

Half-width of each slice, as a fraction of max(|estimate|, 1). Defaults to 0.25.

...

Currently unused.

Details

Every stochastic frontier fit in this package runs a staged minimizer and stores optim()'s convergence code, message, evaluation counts and numerically differentiated Hessian. None of that was reported: a fit that stopped on the iteration cap printed exactly like a converged one. sfa_diagnostics() is the reporting side of that.

What the Hessian tells you. optim() returns the Hessian of the negative log-likelihood, so at a genuine interior optimum it is positive definite. A non-positive eigenvalue means the reported point is not a minimum in that direction. A very small one means the likelihood is nearly flat along it, which is what weak identification looks like numerically – $hessian$flat_direction names the parameters that load on that flattest axis, and they are usually the pair the data cannot separate.

Slices, not profiles. Panel 3 varies one parameter and holds the others at their estimates. A true profile would re-optimize the remaining parameters at every grid point, at the cost of one optimization per point. The slice is the right object for reading curvature at the optimum and for the simplest check there is: the estimate should sit at the peak of its own slice. If it does not, the optimizer stopped early.

Retaining the objective. The gradient and the slices need the likelihood itself, which an ordinary fit does not keep. Pass keep_objective = TRUE to sfm to store it. This is off by default because a closure carries its enclosing environment, so a fit saved with one serializes the estimation data alongside the results – on a 200-observation example the fitted object grows from roughly 38 KB to 1.7 MB.

The convergence code alone is not diagnostic, and the flags do not treat it as though it were. Across NHN, NE and NTN at n=150n = 150, 500500 and 15001500, code 52 (“ABNORMAL_TERMINATION_IN_LNSRCH”) turns up routinely alongside a maximum relative gradient of about 10−610^{-6} and a positive definite Hessian: the staged minimizer had already found the optimum and the final L-BFGS-B stage simply could not take a step away from it. The same code on NTN at n=150n = 150 came with a relative gradient of 5×1075 \times 10^{7} and an indefinite Hessian, which is a genuine failure. What separates the two is the gradient and the Hessian, so the verdict is built from those and the code is reported as context:

benign

a line-search code (51/52) together with a small relative gradient and a positive definite Hessian. Reported as noise.

unverified

a line-search code with no retained objective, so there is no gradient to settle it. Reported as an open question rather than resolved in the fit's favour – refit with keep_objective = TRUE.

failure

anything else, including code 1 (the iteration limit), which is never excused: it means the search was still moving when it ran out of budget, whatever the curvature at the stopping point.

Flags are raised for a non-zero convergence code, a Hessian that is not positive definite or is worse than 10810^{8} conditioned, any parameter pair correlated beyond ∣0.95∣|0.95|, and a large relative gradient. They are phrased as statements about what the fit means, not as generic warnings.

Value

sfa_diagnostics() returns an object of class "sfadiag": a list with model_name, call, pnames (the REPORTED parameter names), enames (the ESTIMATION-scale names, which differ where a fit estimates parameters it does not report, as ivsfm("IVLIML") does), estimates, convergence (code, meaning, message, counts, logLik, objective, objective_type), hessian (eigenvalues, condition, pos_def, flat_direction), correlation (matrix, worst_pair, worst_value); convergence additionally carries benign_nonzero and unverified_nonzero, gradient (gradient, max_abs, relative, max_rel; NULL without a retained objective), flags, and has_objective.

plot.sfareg() is called for its side effect and returns the "sfadiag" object invisibly. print.sfadiag() returns its argument invisibly.

Which scale each diagnostic is on

The Hessian, the gradient and the likelihood slices are indexed by the vector the optimizer worked on, so they are named with enames. The parameter correlation is built from vcov, which is already on the reported scale, so it is named with pnames. Where the two differ, print() says so. convergence$logLik is the log-likelihood and is NA where none is defined (the robust divergence estimators); convergence$objective is the value the optimizer actually achieved, whatever objective that was, and objective_type names it.

Author(s)

David H. Bernstein

See Also

sfm, sfareg-methods, summary.sfareg

Examples

library(sfa)
d <- data_gen_cs(N = 300, rand = 3, sig_u = 1, sig_v = 0.3,
                 cons = 0.5, beta1 = 0.5, beta2 = 0.5, a = 5, mu = 0.5)

## Retaining the objective enables the gradient and the likelihood slices
f <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d, keep_objective = TRUE)
sfa_diagnostics(f)

## Four-panel diagnostic display
plot(f)

## Works without the objective too, on the Hessian alone
g <- sfm(y_pcs ~ x1 + x2, model_name = "NHN", data = d)
sfa_diagnostics(g)$hessian$condition

The wrong-skewness suite: what to do when the residuals lean the wrong way

Description

A guide to the options this package offers when the OLS residuals of a production frontier are skewed positively – the “wrong” way – and the estimated inefficiency collapses to zero. This page is documentation only; each option is a function or an argument documented in its own right.

Details

The problem. Waldman (1982) showed that for the Normal-Half-Normal frontier, if the third central moment of the OLS residuals is positive then the OLS fit is a local maximum of the likelihood at σ^u=0\hat\sigma_u = 0. The fit then reports an industry in which every firm is fully efficient. Horrace and Wright (2020) extend the result to the truncated-normal and doubly-truncated-normal cases. The same thing happens to the moment estimator for a different reason: m3=σu32/π(1−4/π)m_3 = \sigma_u^3\sqrt{2/\pi}(1-4/\pi) has no real solution with σu>0\sigma_u > 0 when m3>0m_3 > 0. Olson, Schmidt and Waldman (1980) call these the Type I failure (no admissible σu\sigma_u) and the Type II failure (no admissible σv\sigma_v).

It is not a data problem. Simar and Wilson (2009) make the point that has to be kept in view: observing wrong skew is entirely consistent with the model being correctly specified. It happens routinely when σu\sigma_u is small relative to σv\sigma_v, and the smaller the sample the more often. So respecifying the model in response to it is, in their phrase, a solution in search of a problem – unless there is an independent reason to prefer the new specification.

What the package offers. They divide into three kinds.

Keep the model, change the estimator.

sfm(estimator = "acols")

Parmeter and Zhao (2023). Corrected OLS with the first absolute moment E∣ε∣E|\varepsilon| in place of the third central moment. The third moment is the only part of the inversion that can fail, so removing it removes the failure: on the authors' own design at λ=0.4\lambda = 0.4, n=200n = 200, Type I failures fall from 48% to 15%.

sfm(estimator = "cmle")

Zhao and Parmeter (2022). Maximum likelihood subject to two moment conditions the assumed density implies. The Waldman stationary point does not satisfy them, so the search is kept away from it. On wrong-skew samples the proportion of fits returning σ^u≈0\hat\sigma_u \approx 0 falls from about 96% to about 3%.

Keep the model, change the question.

skewness_test, symmetry_test

Ask first whether the residual skew is significantly wrong at all, rather than reading its sign off a point estimate. Often it is not, and nothing further is needed.

skewness_decomp

Bonanno, De Giovanni and Domma (2017). Splits the composed third moment into the parts due to the inefficiency, to the noise, and to the dependence between them, and reports Zenga's median-based measure alongside. Under the textbook specification two of the three are identically zero, which is the whole reason a positive residual skew has nowhere to go.

moment_range

What skewness a given pair of distributions can produce at all. A residual skew outside the achievable range is evidence about the distributions, not about the firms.

Change the model.

esfm

Hafner, Manner and Simar (2018). Widens the distribution of uu by a finite-sample correction that admits either sign of skew and converges to the assumed density as nn grows.

copsfm

Bonanno, De Giovanni and Domma (2017), generalizing Smith (2008). A generalized-logistic noise, which can itself be skewed either way, and a copula linking uu and vv. Either ingredient can account for a positive composed skew without setting σu\sigma_u to zero.

sfm(model_name = "NB")

Carree (2002). A binomial inefficiency, the one distribution here that can be skewed either way: p>1/2p > 1/2 means most firms carry considerable inefficiency and only a few sit near the frontier. Same sample moment, opposite economics. Corrected OLS only.

sfm(model_name = "NGB2")

A generalized beta of the second kind inefficiency, whose shape parameters accommodate a much wider range of third and fourth moments than the half-normal.

Choosing. If the skew is not significant, stop. If it is, prefer an estimator change to a model change unless there is a substantive reason for the new distribution, since a respecified model answers a different question. "cmle" and "acols" are the cheapest options and leave the model alone. Among the respecifications, copsfm is the most informative when the mechanism matters, because skewness_decomp then says which ingredient is doing the work.

References

Bonanno, G., De Giovanni, D. and Domma, F. (2017). The “wrong skewness” problem: a re-specification of stochastic frontiers. Journal of Productivity Analysis 47(1), 49-64. doi:10.1007/s11123-017-0492-8

Carree, M. A. (2002). Technological inefficiency and the skewness of the error component in stochastic frontier analysis. Economics Letters 77(1), 101-107. doi:10.1016/S0165-1765(02)00119-2

Hafner, C. M., Manner, H. and Simar, L. (2018). The “wrong skewness” problem in stochastic frontier models: A new approach. Econometric Reviews 37(4), 380-400. doi:10.1080/07474938.2016.1140284

Olson, J. A., Schmidt, P. and Waldman, D. M. (1980). A Monte Carlo study of estimators of stochastic frontier production functions. Journal of Econometrics 13(1), 67-82. doi:10.1016/0304-4076(80)90043-3

Parmeter, C. F. and Zhao, S. (2023). An alternative corrected ordinary least squares estimator for the stochastic frontier model. Empirical Economics 64(6), 2831-2857. doi:10.1007/s00181-023-02401-1

Simar, L. and Wilson, P. W. (2009). Inferences from cross-sectional, stochastic frontier models. Econometric Reviews 29(1), 62-98. doi:10.1080/07474930903324523

Waldman, D. M. (1982). A stationary point for the stochastic frontier likelihood. Journal of Econometrics 18(2), 275-279. doi:10.1016/0304-4076(82)90041-0

Zhao, S. and Parmeter, C. F. (2022). The “wrong skewness” problem: Moment constrained maximum likelihood estimation of the stochastic frontier model. Economics Letters 221, 110901. doi:10.1016/j.econlet.2022.110901

See Also

sfm, copsfm, esfm, skewness_decomp, skewness_test, moment_range


Extractor methods for "sfareg" objects

Description

Standard extractor methods for objects of class "sfareg", which sfm, psfm, zsfm, ttsfm, lcsfm, copsfm, ivsfm, selsfm and esfm return. They let a fit be used with the usual R idioms – coef(fit), vcov(fit), logLik(fit), AIC(fit), BIC(fit) – instead of reaching into the returned list by hand.

method returns
coef the reported parameters, object$coefficients
vcov their covariance: inverse Hessian (default), BHHH, sandwich or clustered
logLik the maximized log-likelihood, with "df" and "nobs" for AIC() and BIC()
nobs the number of observations the fit used
estfun, bread scores and bread for the sandwich package (need keep_objective = TRUE)

AIC() and BIC() are not defined for "sfareg"; the stats defaults work through logLik and nobs.

Usage

## S3 method for class 'sfareg'
coef(object, ...)
## S3 method for class 'sfareg'
vcov(object, type = c("hessian", "bhhh", "sandwich", "clustered"),
  cluster = NULL, ...)
## S3 method for class 'sfareg'
logLik(object, ...)
## S3 method for class 'sfareg'
nobs(object, ...)

Arguments

type

For vcov: "hessian" (the default) for the inverse Hessian, "bhhh" for the inverse outer product of the per-observation scores, "sandwich" for the heteroskedasticity-robust sandwich, or "clustered" for its cluster-robust form. The last three need keep_objective = TRUE; the last two additionally need a Hessian, which "bhhh" does not. See ‘Details’.

cluster

For vcov(type = "clustered"): a vector or factor grouping the ROWS OF THE SCORE MATRIX. That is one row per observation for a cross-sectional fit and one row per FIRM for a panel one, so a firm-year vector is the wrong length by construction and is refused with a message saying so. A data frame is accepted for multiway clustering and is reduced with interaction().

object

An object of class "sfareg".

...

Currently unused; included for S3 method consistency.

Details

What each method returns

coef.sfareg returns the named vector of reported parameters, object$coefficients.

vcov.sfareg with the default type = "hessian" inverts the Hessian stored on the fit (object$opt$hessian). Where the Hessian's diagonal is positive it is first scaled to unit diagonal, so that parameters on very different scales do not make an invertible matrix look singular. Where the fit estimates on a different scale from the one it reports (see ‘The scale the scores live on’ below), the inverse is then mapped onto the reported scale. If the inversion or the mapping fails it warns and falls back to a diagonal matrix built from the reported standard errors (object$std.errors), which it also returns, without a warning, for a fit that stores no Hessian. With neither it returns a matrix of NAs with a warning. The other three types are described below.

logLik.sfareg returns the maximized log-likelihood as a "logLik" object whose "df" and "nobs" attributes make AIC() and BIC() work; see ‘Two quantities that are easy to confuse’ for what "df" counts.

nobs.sfareg returns the number of observations the fit used, not the rows supplied, so BIC() divides by the right nn. It takes the first of: the count stored on the fit (object$nobs); the rows of stored data (object$data); the length of a per-observation vector the fit stores (such as exp_u_hat); and, failing all of those, the rows of the call's data argument re-evaluated in the caller's frames, accepted only if it matches what the fit was made from. It returns NA if none of these is available.

Four covariances

vcov(type = "sandwich") returns the heteroskedasticity-robust sandwich and vcov(type = "clustered") its cluster-robust form. Both are formed on the ESTIMATION scale – the scale the scores and the Hessian live on – and only then mapped to the scale coef reports. That mapping is why they are not left to sandwich::sandwich(): that function composes bread and meat itself and has no way to know the two scales differ. For ttsfm, copsfm, ivsfm and psfm's PL80 family they do differ, and for ivsfm("IVLIML") they differ in dimension as well, since it maximises over 11 parameters and reports 6.

sandwich::sandwich() and sandwich::vcovCL() remain available and now return an estimation-scale covariance, which is internally consistent but not on the scale coef reports. Where a model reports what it estimates – sfm, zsfm, lcsfm, psfm's TRE/GTRE family – the two routes agree exactly, and the cluster adjustment matches vcovCL's default of M/(M−1)M/(M-1).

BHHH standard errors

vcov(fit, type = "bhhh") returns the inverse of the outer product of the per-observation scores instead of the inverse Hessian. Its value is that it needs no Hessian at all: it is defined whenever the scores are, which is exactly the case where the default path fails. When the Hessian is singular vcov otherwise falls back to a diagonal approximation built from the reported standard errors, discarding every covariance; BHHH keeps them. It requires keep_objective = TRUE for the same reason estfun does, and is returned on the same scale coef reports (see below). It is not the default, because it is only as good as the information-matrix equality and so disagrees with the Hessian under misspecification – which is useful when the two are compared on purpose and misleading when they are not. At a parameter bound the scores in the constrained directions are not defined: the likelihood returns a stability penalty rather than a value, so estfun falls back to a one-sided difference on the admissible side, uses zero where neither side is admissible, and warns. BHHH errors built from such a fit understate the uncertainty in those directions.

Robust and clustered standard errors with the sandwich package

vcov returns the inverse Hessian, which is valid only if the likelihood is correctly specified and the observations are independent – neither being safe to assume in applied frontier work. Registering bread and estfun makes the sandwich package operate on these fits, so sandwich::sandwich() gives Huber–White errors and sandwich::vcovCL() gives clustered ones, and either can be handed to lmtest::coeftest():

fit <- sfm(y ~ x1 + x2, model_name = "NHN", data = d,
           keep_objective = TRUE)
lmtest::coeftest(fit, vcov. = sandwich::vcovCL(fit, cluster = d$firm))

Two requirements. The fit must have been made with keep_objective = TRUE, since the scores are differenced from the retained likelihood; and the robust divergence estimators are excluded, because they do not maximise a log-likelihood and so have no score. For psfm panel fits the scores are per firm, so BHHH and sandwich standard errors are clustered by firm, and bread() scales by the number of firms rather than by nobs(). sandwich::vcovHC() does not work and is not expected to: its finite-sample corrections are built from hat values, which are not defined for a nonlinear likelihood whose parameter vector includes variance components.

Which fits can supply scores

keep_objective = TRUE is available on every entry point whose likelihood is separable into per-observation contributions: sfm, zsfm, lcsfm, ttsfm ("TTNE" and "TTHN"), copsfm, ivsfm ("IVLIML" and "IVCF"), selsfm, and psfm for "GTRE" (estimator = "sml"), "TRE", "GTRE_Z", "TRE_Z", "GTRE_FML", "TFE_WMLE", "FD", "PL80", "PL80_MVTN", "BC92", "K1990" and "K1990modified", whose contributions are per firm. All but sfm and psfm's four simulated-ML models are new in 1.2.1. The estimators that maximise no log-likelihood have no score and say so rather than returning an unusable fit: ttsfm("TTNLS") is nonlinear least squares, ivsfm("C2SLS") is 2SLS with a corrected intercept, psfm's "TFE", "SSFE", "SSRE", "SSCRE", "CSS", "LS", "KSS", "GTRE_SEQ1" and "GTRE_SEQ2" are moment-based, FE/LSDV or sequential, and npsfm is nonparametric.

The scale the scores live on

estfun differences the retained likelihood at fit$opt$par, which is not always what coef reports: ttsfm, copsfm and ivsfm estimate log sigmas and report sigmas, ivsfm reports ρ=t/1+t′t\rho = t/\sqrt{1 + t't}, and ivsfm("IVLIML") additionally estimates reduced-form parameters it never reports. vcov therefore carries the fit's own Jacobian from the estimation scale onto the reported one – the same delta method the reported standard errors already use – and marginalises any parameters that are estimated but not reported by taking the corresponding block of the inverse, not the inverse of the block. estfun itself returns the scores on the estimation scale, with that map attached as attributes, so its column count can exceed length(coef(fit)). From 1.2.1 this correction is applied on the Hessian path too: before it, sqrt(diag(vcov(f))) on a copsfm fit disagreed with the standard errors the same fit printed by a factor of 1/σ^u1/\hat\sigma_u, and on an ivsfm("IVLIML") fit vcov failed outright.

The scores are taken by central differences with a per-parameter step, there being no analytic gradient for most of these models. Where one does exist (NHN) the two agree to about 3×10−73 \times 10^{-7}, and they agree with numDeriv to the same order.

Two quantities that are easy to confuse

object$opt$value is the OBJECTIVE the optimizer minimized, which is the negative summed log-likelihood for most but not all estimators here, and object$coefficients is what the fit REPORTS, which is not always what it estimated. logLik() therefore reads neither blindly: "df" is the number of parameters ESTIMATED (length(object$opt$par) where that exists), because ivsfm("IVLIML") maximizes over 11 and reports 6; and the returned value is the PLAIN log-likelihood, because lcsfm with penalty_c > 0 maximizes a penalized likelihood and carries the unpenalized one separately.

Estimators with no log-likelihood

sfm's robust divergence estimators (robust = "mlqe", "psi" or "mdpd") minimize a divergence rather than a negative log-likelihood, so logLik() returns NA with a warning and AIC()/ BIC() propagate it. Their objective is not on the likelihood scale and must not be compared with one; print() names it accordingly. Exception: psfm()'s "GTRE_SEQ1" and "GTRE_SEQ2" models are estimated by a moment-based procedure and "SSFE" by fixed-effects (LSDV) regression, not by maximum likelihood; none of the three carries an $opt component, so calling logLik() (or, transitively, AIC()/BIC()) on those fits returns NA with a warning rather than erroring.

Value

coef.sfareg returns a named numeric vector.

vcov.sfareg returns a numeric matrix.

logLik.sfareg returns an object of class "logLik".

nobs.sfareg returns a single integer (or NA).

See Also

print.sfareg, summary.sfareg, TIC, influence_sfa; the entry points listed under ‘Description’.


Cross-Sectional Stochastic Frontier Models

Description

Implementation of the cross-sectional stochastic frontier model across an array of distributional assumptions for both v and u (user specified). For panel models, see the psfm() call.

Usage

sfm(formula, model_name = c("NHN", "NHN_Z", "NE", "NE_Z", "NR",
    "THT", "NTN", "NG", "NNAK", "NU", "NGE", "NLN", "NW", "tHN",
    "TSL", "NB", "NGB2"), data, maxit.bobyqa = 10000, maxit.nlminb = 500,
    maxit.psoptim = 1000, maxit.optim = 1000, REPORT = 1, trace = 2,
    pgtol = 0, start_val = FALSE, PSopt = FALSE, use.nlminb = "auto",
    use.bobyqa = "auto", optHessian = TRUE, inefdec = TRUE, upper =
    NA, Method = "L-BFGS-B", robust = c("mle", "mlqe", "psi",
    "mdpd"), c_mlqe = 0.2, eta = 0.01, alpha = 0.2, verbose = FALSE,
    start_from = NULL, Nsim = "auto", z_link = c("sd", "var"), vhet =
    NULL, uhet = NULL, muhet = NULL, scaling = NULL, shapehet = NULL,
    weights = NULL, wscale = TRUE, sim_type = c("halton", "sobol",
    "torus", "uniform"), antithetics = FALSE, sim_burn = NULL,
    sim_scrambling = 0L, sim_prime = NULL, sim_seed = NULL,
    rand.psoptim = NULL, keep_objective = FALSE, estimator = c("mle",
    "cols", "mols", "acols", "cmle"), cols_boot = 0, rand.cols = NULL)

Arguments

formula

a symbolic description for the model to be estimated

model_name

Which distributional pair to fit. The noise term vv is normal except where stated, and the one-sided inefficiency term uu follows:

"NHN"

half-normal.

"NHN_Z"

half-normal with heteroskedastic σu\sigma_u, parameterized by the determinants zz given in the second part of the formula (y ~ x | z) as σu=exp⁡(z′δ)\sigma_u = \exp(z'\delta).

"NE"

exponential.

"NE_Z"

exponential with heteroskedastic σu\sigma_u, parameterized as for "NHN_Z".

"NTN"

truncated normal.

"NR"

Rayleigh. Note this is a distinct family from "NHN", not a reparameterization of it: the Rayleigh density carries an extra factor of uu, and its standardized skewness is fixed at -0.631 against the half-normal's -0.995.

"NU"

uniform.

"NG"

gamma. See ng_starts under ‘Value’ – this likelihood determines E[u]=μσuE[u] = \mu\sigma_u far better than either factor separately.

"NNAK"

Nakagami, with shape mm. Nests "NR" at m=1m = 1. The shape is weakly identified in practice, so σu\sigma_u and mm are strongly correlated across samples.

"NGE"

generalized exponential.

"NLN"

lognormal. Estimated by simulated maximum likelihood; see Nsim and ‘Details’.

"NW"

Weibull. Also simulated maximum likelihood.

"tHN"

half-normal, with Student-t noise.

"TSL"

truncated skew-Laplace.

"THT"

half t, with Student-t noise.

"NGB2"

generalized beta of the second kind (Makiela and Mazur 2022), with scale σu\sigma_u and three shape parameters (ν,ψ,τ)(\nu, \psi, \tau). The most general one-sided family here: it nests "NHN", "NE", "NG" and "NW" as limits and the half-Student tt exactly. Evaluated by deterministic quadrature, not simulation; see ‘Details’.

"NB"

binomial (Carree 2002). Corrected OLS only – call it with estimator = "cols"; sfm() refuses it under maximum likelihood. The only inefficiency distribution here that can be skewed either way, which is the whole reason for it: see ‘Details’.

Matching ignores case, so "nhn", "NHN" and "Nhn" are the same choice.

data

A data frame containing the variables named in formula.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine

maxit.optim

Maximum number of iterations for the optim optimization routine

REPORT, trace, pgtol, upper

Accepted but not used by sfm(): the optimizer's own tracing is controlled by verbose, and its bounds are set internally.

start_val

starting value (optional)

PSopt

use psoptim optimization routine (T or F)

maxit.nlminb

Maximum number of iterations for the nlminb optimization routine.

use.nlminb

"auto" (default), or a logical. Whether to run the stats::nlminb stage, which precedes bobyqa in the optimizer stack and accepts an analytic gradient where one is available. Under "auto" it runs only for the models where it has been verified to reach an equal-or-better optimum than the previous bobyqa-based path (NHN, NE, NTN, NU), where it is roughly 4-8 times faster at identical accuracy.

use.bobyqa

"auto" (default), or a logical. Whether to run the derivative-free bobyqa stage. Under "auto" it runs for every model except those handled by nlminb above, reproducing the previous behaviour exactly. It is kept on by default for the remaining models deliberately: for NR and NGE the nlminb path converges to a materially worse optimum, so accuracy, not speed, decides which stage runs.

optHessian

Logical. Should a numerically differentiated Hessian matrix be returned while using the optim routine? (for optim routine)

inefdec

Production or cost function

Method

Method passed to optim for the final stage. Must be "L-BFGS-B": it is the only optim method that applies the lower and upper bounds this package uses to keep scale parameters positive, and any other value is refused with an error rather than quietly optimized without them.

robust

One of "mle" (default), "mlqe", "psi", or "mdpd". Selects a robust divergence-based objective in place of ordinary maximum likelihood – see Details. Currently only implemented for model_name = "NHN"; sfm() stops with a clear error if a non-"mle" value is requested for any other model.

c_mlqe

Numeric, default 0.20. The MLqE power parameter (c in f^c; the corresponding q = 1-c), used only when robust = "mlqe".

eta

Numeric, default 0.01. The Psi-likelihood power parameter, used only when robust = "psi".

alpha

Numeric, default 0.2. The MDPD (minimum density power divergence) power parameter, used only when robust = "mdpd".

verbose

Logical. Print optimization progress messages? Default is FALSE.

weights

Optional observation weights, one per row of the data actually fitted (that is, after missing-data handling; a length mismatch is an error rather than being recycled). They multiply the per-observation log-likelihood contributions. For frequency weights this is exact – a weight of 2 gives the same estimates as including the row twice, which the package tests – and for sampling weights it yields a pseudo-likelihood whose point estimates are consistent but whose Hessian-based standard errors are not. In that case use the sandwich methods this package registers, e.g. sandwich::sandwich(fit). Cannot be combined with the robust divergence estimators, which reweight observations themselves.

wscale

Rescale weights to sum to the number of observations. This puts the log-likelihood on the same scale as an unweighted fit, so AIC/BIC stay comparable; it is a common factor on the objective and does not move the estimates. Set FALSE for genuine frequency weights, where the unscaled total is the quantity of interest.

shapehet

Experimental. Optionally, a one-sided formula parameterizing the shape of uu rather than its scale, as μi=μexp⁡(zi′δ)\mu_i = \mu\exp(z_i'\delta). Implemented for model_name = "NG" and "NNAK", the two families that have a shape parameter at all. On a design with a true shape effect it improves the log-likelihood by a very large margin (about 10,000 points at n=2000n = 2000), so it does capture the variation. But its parameters do not recover well in testing: on that design the shape coefficient came back at 0.19 against a true 0.4 while σV\sigma_V collapsed to its lower bound, which is the signature of a degenerate optimum rather than an estimate. Plain "NG" recovers cleanly on the same design without a shape effect, so this is specific to the extended model and not to "NG" itself. Treat the coefficients as exploratory, check σV\sigma_V for a boundary collapse before reading them, and prefer uhet where scale heterogeneity is what you actually mean.

scaling

Optionally, a one-sided formula of covariates imposing the scaling property of Wang and Schmidt (2002) and Alvarez, Amsler, Orea and Schmidt (2006): ui=h(zi,δ) ui∗u_i = h(z_i,\delta)\,u^*_i with h=exp⁡(zi′δ)h = \exp(z_i'\delta), so the covariates scale a common draw u∗u^* rather than entering its variance. Both σU\sigma_U and the pre-truncation mean move by the same factor, which holds the shape of the distribution fixed and leaves only its scale free – the identification argument those papers make. Implemented for model_name = "NTN" only: for a half-normal uu, h(z)∣N(0,σ2)∣h(z)|N(0,\sigma^2)| is ∣N(0,(hσ)2)∣|N(0,(h\sigma)^2)|, so uhet already fits that model exactly and a separate option would be two names for one thing. Cannot be combined with uhet or muhet, since letting those vary separately would undo the restriction. The reported scale. coefficients are δ\delta; Zu.(Intercept) and Zmu.(Intercept) are the scale and pre-truncation mean of the common draw.

start_from

Optionally, a previously fitted "sfareg" object to take starting values from. Parameters are matched by name, not by position: "NE" reports sigv/sigu where "NHN" reports lambda/sigma, so a positional copy would slide a variance ratio into a standard deviation. Anything unmatched keeps the start the package computes for it, and a fit sharing nothing but the frontier coefficients carries only those. Fitting a simpler model first and cascading is the documented remedy for the models with awkward likelihoods: over forty samples for "NG" seeded from "NE", the seeded fit was never worse, indistinguishable on thirty-nine, and worth 176.7 log-likelihood points on one. The rescue is rare and large rather than common and small, which is what makes it worth doing: the sample it rescues is one where the default start converges quietly to a much worse optimum, with nothing in the returned object to say so.

Nsim

Total number of simulation draws per observation for the simulated-maximum-likelihood models ("NLN", "NW"); ignored by every closed-form model. "auto" (the default) uses max(200, ceiling(3*sqrt(n))). Half the draws are taken from each of the two proposals described under ‘Details’. Simulated ML is consistent only if the draw count grows with the sample size – at a fixed Nsim the simulation bias does not vanish and every parameter except the frontier slopes converges to the wrong value. The previous fixed default of 100 did exactly that: at n=3000n = 3000 it returned (0.362, 0.821, 1.284, 0.328) against a truth of (0.3, 1.0, 1.5, 0.5) for "NW". A numeric value below the automatic floor warns. Earlier versions warned that "NLN" needed far more draws than "NW"; that was a property of the old integration scheme, not of the model. Both now behave alike: at n=3000n = 3000, measured against quadrature at the true parameters, total simulation error is 3.4 log-likelihood units for "NLN" and 3.0 for "NW" at the default, falling to 0.07 and 0.06 at Nsim = 6400. The comparable figure for the old scheme was 226.8.

rand.psoptim

Integer. seed for replication of psoptim. Default to NULL.

keep_objective

Logical. Store the likelihood function on the fitted object, so that sfa_diagnostics can compute the gradient at the optimum and draw likelihood slices. Defaults to FALSE: a closure carries its enclosing environment, so a fit saved with one serializes the estimation data too – on a 200-observation example the object grows from about 38 KB to 1.7 MB. Everything else sfa_diagnostics() reports works without it.

estimator

"mle" (default), or "cols"/"mols" – two names for the same thing, see ‘Details’ – which select the modified ordinary least squares moment estimator instead of maximum likelihood: OLS slopes are kept as they are, the second and third central moments of the OLS residuals are inverted for the scale parameters, and the intercept is shifted up by the implied E[u]E[u]. Closed form, no optimizer, deterministic. Implemented for "NHN", "NE" and "NG"; other models error, because the moment inversion is distribution-specific.

Two further values belong to the wrong-skewness suite (see sfa-wrongskew) and are implemented for "NHN" and "NE" only:

"acols"

The alternative corrected OLS of Parmeter and Zhao (2023). Identical to "cols" except that the FIRST ABSOLUTE moment E∣ε∣E|\varepsilon| replaces the third central moment. The third moment is what fails in small samples – it is the only ingredient that can drive σu\sigma_u to zero – and E∣ε∣E|\varepsilon| carries the same information about the mix without cubing the residuals. There is no closed form, so the moment equation is solved numerically. Standard errors need cols_boot: the analytic errors reported for "cols" are Coelli's delta method for the third-moment inversion and do not apply.

"cmle"

The moment-constrained maximum likelihood of Zhao and Parmeter (2022). Ordinary maximum likelihood subject to two equality constraints that the assumed density implies: E∣ε∣=σ2/πE|\varepsilon| = \sigma\sqrt{2/\pi} and Var(ε)=σv2+π−2πσu2\mathrm{Var}(\varepsilon) = \sigma_v^2 + \frac{\pi-2}{\pi}\sigma_u^2 for "NHN", and the analogues of their Proposition 2.1 and Eq. (2.8) for "NE". The constraints keep the search away from the Waldman (1982) boundary at which OLS is a stationary point of the likelihood, without changing the model. Nothing is respecified: the density is the same one estimator = "mle" maximizes, and the two estimators are asymptotically identical.

See ‘Details’.

z_link

Scale on which the variance-determinant linear predictor is placed, for the _Z models. "sd" (the default, and sfm's historical convention) sets σu=exp⁡(z′δ)\sigma_u = \exp(z'\delta); "var" sets σu=exp⁡(z′δ)\sigma_u = \sqrt{\exp(z'\delta)}, which is what psfm and the competing packages use. The two are reparameterizations of the same model – they reach the same maximised log-likelihood and the same marginal_effects – but δ\delta itself differs by a factor of two, so a δ\delta from sfm is not comparable with one from psfm or from sfaR unless the same convention is used. Set z_link = "var" to put them on one footing. See ‘Details’.

vhet

One-sided formula for the determinants of the noise scale, e.g. vhet = ~ z1 + z2, giving σv=exp⁡(zv′δv)\sigma_v = \exp(z_v'\delta_v) under the SD link. NULL (the default) leaves σv\sigma_v constant. Available for "NHN", "NHN_Z", "NE", "NE_Z" and "NTN".

uhet

One-sided formula for the determinants of σu\sigma_u. This is the same specification as the | z segment of formula, and it is an error to give both; uhet exists so that "NTN", which takes no pipe segment, can still have a heteroskedastic σu\sigma_u.

muhet

One-sided formula for the determinants of the pre-truncation mean of uu, μ=zμ′δμ\mu = z_\mu'\delta_\mu. This is the Battese and Coelli (1995) inefficiency-effects model and requires model_name = "NTN", the only family in which μ\mu exists. Unlike the two scale blocks μ\mu carries no link, since it may take either sign.

sim_type

Sequence used to build the simulated-ML draws, for the models fitted by simulation ("NLN", "NW"). "halton" (default) reproduces previous behaviour. "sobol" accepts sim_scrambling 1–3, giving the scrambled sequence of Bhat (2003). "torus" is the Korobov sequence and accepts sim_prime. "uniform" is pseudorandom, present as a baseline to measure against rather than as a recommendation. See ‘Details’.

antithetics

Logical. If TRUE, half the draws are taken from the sequence and the other half created as their mirror images (Hammersley and Morton 1956). Default FALSE.

sim_burn

Number of leading elements of the sequence discarded. Default NULL, meaning 1000. The initial elements of different sequences are correlated with one another and discarding them removes it.

sim_scrambling

Integer 0–3, sim_type = "sobol" only, passed to randtoolbox::sobol. Non-zero gives a scrambled sequence.

sim_prime

Prime for sim_type = "torus". Default NULL uses the generator's own choice.

sim_seed

Seed for the draws, for reproducibility. The caller's random-number stream is restored afterwards.

cols_boot

Integer. Number of nonparametric bootstrap resamples used to obtain standard errors under estimator = "cols" or "acols". Zero (the default) skips the bootstrap, in which case only the OLS slope standard errors are reported and the scale parameters and corrected intercept carry NA. Ignored when estimator = "mle".

rand.cols

Integer. Seed for the COLS bootstrap, for reproducibility. The caller's random-number stream is restored afterwards. Default NULL.

Details

sfm() fits a cross-sectional frontier y=x′β+v−uy = x'\beta + v - u (production; +u+u for a cost frontier) by maximum likelihood unless stated otherwise. Pick the pair of distributions for the noise vv and the inefficiency u≥0u \ge 0:

model_name noise vv / inefficiency uu use it when
"NHN" normal / half-normal the default; the standard frontier
"NE" normal / exponential a common alternative to "NHN"
"NTN" normal / truncated normal inefficiency with a non-zero mode; with muhet, Battese-Coelli (1995)
"NHN_Z", "NE_Z" as above, σu=exp⁡(z′δ)\sigma_u = \exp(z'\delta) covariates drive the scale of inefficiency
"NG", "NNAK" normal / gamma, Nakagami a flexible shape for uu
"NR", "NU", "NGE", "TSL" normal / Rayleigh, uniform, generalized exponential, truncated skew-Laplace alternative one-sided shapes
"NLN", "NW" normal / lognormal, Weibull simulated maximum likelihood
"NGB2" normal / generalized beta II one fit that nests most of the above
"tHN", "THT" Student tt / half-normal, half tt heavy-tailed noise
"NB" normal / binomial wrong skewness read as widespread inefficiency; moments only

Each is described in ‘Notes on individual models’ below. Other estimators (estimator = "cols", "acols", "cmle") and the robust divergences (robust) are described in their own subsections.

Estimators other than maximum likelihood

Modified ordinary least squares (estimator = "cols" or "mols"). The moment estimator of Olson, Schmidt and Waldman (1980, Journal of Econometrics), standardly called MOLS. Note the name: this is not Winsten's corrected OLS, which shifts the intercept by the largest residual and estimates no variance parameters at all. The package has shipped this estimator under the spelling "cols" since it was introduced, so that spelling continues to work and is not deprecated; "mols" is accepted as an exact synonym so the method can be found under its usual name. OLS is consistent for the slopes of a composed-error frontier whatever the one-sided distribution; only the intercept is biased, by E[u]E[u], because the composed error does not have zero mean. COLS therefore keeps the OLS slopes, inverts the central moments of the OLS residuals for the scale parameters, and shifts the intercept up by the implied E[u]E[u]. Writing m2m_2, m3m_3 for the second and third central moments of e=v−ue = v - u: for "NHN", m3=σu32/π(1−4/π)m_3 = \sigma_u^3\sqrt{2/\pi}(1-4/\pi) and m2=σv2+σu2(1−2/π)m_2 = \sigma_v^2 + \sigma_u^2(1-2/\pi) with E[u]=σu2/πE[u] = \sigma_u\sqrt{2/\pi}; for "NE", m3=−2σu3m_3 = -2\sigma_u^3 and m2=σv2+σu2m_2 = \sigma_v^2+\sigma_u^2 with E[u]=σuE[u] = \sigma_u; for "NG", from the third and fourth cumulants, σu=−k4/(3k3)\sigma_u = -k_4/(3k_3) and μ=−k3/(2σu3)\mu = -k_3/(2\sigma_u^3) with E[u]=μσuE[u] = \mu\sigma_u.

No optimizer runs and the answer is deterministic, which makes COLS a natural robustness check against a maximum-likelihood fit that may have settled at a local optimum. It is less efficient than ML when the distributional assumption holds, and the "NG" inversion in particular leans on a fourth cumulant and is noticeably noisier than the other two at moderate sample sizes.

Wrong skew under COLS. A production frontier implies m3<0m_3 < 0. When a sample comes out with m3≥0m_3 \ge 0 the moment equations have no admissible solution: sfm() warns, reports sigu as 0 with the whole residual variance assigned to sigv, and returns no efficiency predictions. Read that as no evidence of inefficiency in these data, not as an estimate of zero. This is the Type I failure of Olson, Schmidt and Waldman and is common in small samples; $wrong_skew and $residual_moments record it.

The wrong-skewness suite (estimator = "acols" and "cmle"). Both address the same defect from opposite directions, and neither changes the model. Waldman (1982) showed that when the OLS residuals of a Normal-Half-Normal frontier are skewed the wrong way – positively, for a production frontier – the OLS fit is a local maximum of the likelihood at σ^u=0\hat\sigma_u = 0, so the estimate says there is no inefficiency anywhere in the industry. Nothing is wrong with the data when this happens: Simar and Wilson (2009) point out that wrong skew arises routinely when σu\sigma_u is small relative to σv\sigma_v, entirely within the assumed model. "acols" removes the third moment from the moment estimator, since that is the only ingredient which can produce the boundary; "cmle" keeps the likelihood but adds two constraints the assumed density implies, which the boundary does not satisfy. See sfa-wrongskew for the other options the package offers and how they differ.

Heteroskedasticity

The variance-determinant scale. sfm's _Z models place z′δz'\delta on the standard deviation, σu=exp⁡(z′δ)\sigma_u = \exp(z'\delta), while psfm's TRE_Z/GTRE_Z place it on the variance, σu=exp⁡(z′δ)\sigma_u = \sqrt{\exp(z'\delta)}, as do the competing packages. Since exp⁡(η)=exp⁡(η/2)2\exp(\eta) = \exp(\eta/2)^2, fitting the same data under the two conventions returns the same σu\sigma_u, the same log-likelihood and the same marginal effects, but δ\delta under the SD link is exactly half δ\delta under the variance link. Reading a δ\delta from one family as though it came from the other therefore doubles or halves every reported effect. z_link exists so the two can be aligned; the default preserves sfm's own convention so existing results do not move.

Heteroskedasticity in more than one component. vhet, uhet and muhet let σv\sigma_v, σu\sigma_u and (for "NTN") the pre-truncation mean μ\mu each depend on their own covariates, in any combination. They are named formulas rather than further pipe segments deliberately: pipe position already carries different meanings in different model families – the second segment is σw\sigma_w in ttsfm but σh\sigma_h in psfm's GTRE_Z – and a fourth position would not be readable. The existing | z segment keeps its meaning, so nothing that worked before changes.

Two reasons to reach for vhet in particular. Ignored heteroskedasticity in vv does not merely cost efficiency: Caudill, Ford and Gropper (1995) show that it biases the estimated frontier, because the composed error's mean depends on the scales. And a σv\sigma_v that varies with observed characteristics is one of the few specifications that can absorb apparent wrong skewness without forcing σu\sigma_u onto the zero boundary. muhet with model_name = "NTN" is the Battese and Coelli (1995) inefficiency-effects model, in which covariates shift the location of inefficiency rather than its dispersion; that is usually the model wanted when the question is “what makes firms inefficient”, since δμ\delta_\mu then has a sign that reads directly.

These fits carry every positive quantity on the log scale, so the parameter vector is unconstrained and no variance can be driven onto a boundary by the optimizer; that is also why this path uses nlminb followed by an unbounded BFGS rather than the bounded four-stage scaffold the other models use. Coefficients are reported prefixed by block – Zv., Zu., Zmu. – since the same covariate may appear in more than one. marginal_effects reads the σu\sigma_u block of these fits with no extra arguments. estimator = "cols" and the robust divergences are moment-based and homoskedastic respectively, so neither combines with these arguments; both refuse rather than silently ignoring them.

Efficiency predictions

Outputs include E[exp(-u)|e] given by exp_u_hat, following Battese and Coelli (1988, JoE), where appropriate. As of this version exp_u_hat is also returned for NE, NTN, NU, NGE, NLN and NW, together with the Jondrow et al. (1982) point predictor u_hat = E[u|e]; previously the exponential and truncated-normal models returned no efficiency prediction at all.

Notes on individual models

NR is not NHN. "NR" places a Rayleigh distribution on uu, not a half-normal, and the two are separate families rather than reparameterizations of one another: the Rayleigh density carries an extra factor of uu, and no rescaling removes it. The practical consequence is that the standardized skewness each contributes to the composed error is a different constant, −0.631-0.631 against the half-normal's −0.995-0.995, so a sample generated under one cannot be fitted by the other. σu\sigma_u is reported on the second-raw-moment convention E[u2]=σu2E[u^2] = \sigma_u^2, matching "NHN" and making the two directly comparable; the Rayleigh scale is then σu/2\sigma_u/\sqrt{2}, with E[u]=σuπ/2E[u] = \sigma_u\sqrt{\pi}/2 and Var(u)=(1−π/4)σu2Var(u) = (1-\pi/4)\sigma_u^2. Starting values come from inverting these moments, since the Rayleigh skewness is a constant and so m3m_3 identifies Var(u)Var(u) outright. Use data_gen_cs()'s y_pcs_r column to test it, not y_pcs.

tHN is not THT. In THT (Tancredi 2002) a single scale mixture is shared by both error components, so the noise and the inefficiency are both Student's t with the same degrees of freedom, and that shared mixture is what makes the composed error a closed-form skew-t. In tHN the noise is Student's t and the inefficiency is half-normal, drawn independently, so the two components have different tail behaviour, there is no closed form, and the density is the convolution f(e)=∫0∞fv(e+u)fu(u)duf(e) = \int_0^\infty f_v(e+u) f_u(u) du evaluated by Gauss-Legendre quadrature. tHN is the heavy-tailed-noise model, which is what makes it the natural parametric comparison for the density-power robust estimators (robust = "mlqe", "psi", "mdpd"); THT cannot play that role because its inefficiency term is heavy-tailed too. Parameters are reported as (sigv, sigu, nu), the conventional order, not THT's inverted (sigu, sigv, a).

Two properties of tHN are worth knowing before using it. First, the degrees of freedom are weakly identified: on data simulated from the model at n=1000n = 1000 with a true ν=5\nu = 5, the profile log-likelihood varies by only about 0.24 across ν\nu from 10 to 100 and peaks near 20, so a single reported ν\nu should not be taken at face value. Profile over a grid of fixed ν\nu and report the path. Because of that flat ridge sfm() always fits tHN from several widely separated starting values, keeps the best, and records the outcome in thn_starts; it warns when they reach different optima. Second, on real data σu\sigma_u can collapse onto zero, the heavy noise tail absorbing the entire one-sided component and leaving mean predicted efficiency near one. That is a property of the model rather than a numerical failure, so it is reported rather than suppressed: sfm() warns and sets thn_sigma_u_at_bound. Fits are also substantially slower than the closed-form models, since every likelihood evaluation is a quadrature whose node count grows with σu/σv\sigma_u/\sigma_v.

"NU" assumes u∼U(0,θ)u \sim U(0,\theta) (Li 1996; Nguyen 2010), giving the closed-form composed density f(e)=θ−1[Φ((e+θ)/σv)−Φ(e/σv)]f(e) = \theta^{-1}[\Phi((e+\theta)/\sigma_v) - \Phi(e/\sigma_v)]. The reported theta is the upper bound of the inefficiency support, so predicted inefficiency is bounded by construction. "NGE" assumes a generalized exponential F(u)=(1−e−λu)2F(u) = (1-e^{-\lambda u})^2, whose composed density is a difference of two exponentially-tilted normal tail terms; it nests the exponential model and permits a non-monotonic inefficiency density. NGE reports sigu on the same scale as NE (the exponential mean, λ=1/σu\lambda = 1/\sigma_u) so the two are directly comparable.

"NLN" (uu lognormal, reported as sigu = sdlog and mu = meanlog) and "NW" (uu Weibull, reported as sigu = scale and k = shape) have no closed-form composed density and are estimated by simulated maximum likelihood. The integral f(e)=∫0∞fu(u) ϕ((e+u)/σv)/σv duf(e) = \int_0^\infty f_u(u)\,\phi((e+u)/\sigma_v)/\sigma_v\,du is a product of two densities, and simulating from either one alone fails wherever the other is the narrow factor: draws from fuf_u miss the normal kernel for a very inefficient unit, while draws from the kernel miss a sharply peaked fuf_u. Half the draws are therefore taken from each, and combined by the balance heuristic of Veach and Guibas (1995), which weights every draw by the proposal likelier to have produced it. This needs no rule for choosing between the two and remains accurate in both regimes. The draws are generated once and reused at every optimizer iteration, so the objective is deterministic and reproducible; increase Nsim to reduce simulation error.

The efficiency predictor reuses the weights the likelihood computed, so u_hat is the Bayes rule under exactly the density that was maximised.

Simulated-ML draws. "NLN" and "NW" have no closed-form composed density and are fitted by maximum simulated likelihood, averaging over draws built by sim_type. Two properties are enforced whatever the sequence. Each observation receives its own contiguous block of the sequence rather than a strided subsequence: a column-major fill hands observation ii the elements hi,hi+n,…h_i, h_{i+n}, \ldots of a van der Corput sequence, which is not equidistributed – at n=500n = 500 and Nsim = 100 the first observation's block spanned only [0.50,0.75][0.50, 0.75], integrating over a quarter of the inefficiency distribution. And the draws are clamped away from 0 and 1, since qnorm() of an exact endpoint is infinite.

Blocking by observation is also what makes the draws worth using: Train (2002, ch. 9) attributes the advantage of a low-discrepancy sequence to its coverage and to the negative correlation it induces across observations, and the second only arises when observations receive different blocks. On draw counts, Bhat (2001) found 100 Halton draws more precise than 1000 pseudorandom ones, with 125 Halton draws giving half the simulation error of 1000 random draws; Nsim = "auto" scales the count with nn and warns if a supplied count is too small.

"NGB2": the generalized beta of the second kind. Makiela and Mazur (2022), section 3, with density

f(u)=ψuσuB(τ/ψu,νu/ψu) νuτ/ψu(uσu)τ−1[1+1νu(uσu)ψu]−(τ+νu)/ψu,f(u) = \frac{\psi_u}{\sigma_u B(\tau/\psi_u, \nu_u/\psi_u)\,\nu_u^{\tau/\psi_u}} \left(\frac{u}{\sigma_u}\right)^{\tau-1} \left[1 + \frac{1}{\nu_u}\left(\frac{u}{\sigma_u}\right)^{\psi_u}\right]^{-(\tau+\nu_u)/\psi_u},

reported as (sigv, sigu, nu, psi, tau). σu\sigma_u is a scale, νu\nu_u governs tail thickness (and so which moments exist, like the degrees of freedom of a tt), and ψu\psi_u and τ\tau are shape.

The point of four parameters on one error component is what they nest. The limits are most of the rest of this table, so one fit says which of them the data want rather than a column of separate fits saying it by their likelihoods:

restriction distribution
νu→∞\nu_u \to \infty generalized gamma
νu→∞\nu_u \to \infty, τ=1\tau = 1, ψu=2\psi_u = 2 half-normal ("NHN")
νu→∞\nu_u \to \infty, τ=ψu=1\tau = \psi_u = 1 exponential ("NE")
νu→∞\nu_u \to \infty, ψu=1\psi_u = 1 gamma ("NG")
νu→∞\nu_u \to \infty, τ=ψu\tau = \psi_u Weibull ("NW"), at scale σuψu1/ψu\sigma_u\psi_u^{1/\psi_u}
τ=1\tau = 1, ψu=2\psi_u = 2 half Student tt with νu\nu_u degrees of freedom
τ=ψu=1\tau = \psi_u = 1 Lomax (Pareto II), shape νu\nu_u and scale σuνu\sigma_u\nu_u

The half-tt and Lomax cases are exact; the others are limits, and the tests check that the error against each falls like 1/νu1/\nu_u rather than merely that it is small at one value.

Note how the Lomax row sits against the exponential row above it: they carry the same restriction τ=ψu=1\tau = \psi_u = 1 and differ only in whether νu\nu_u is sent to infinity or left free, exactly as the half-tt row is the finite-νu\nu_u version of the half-normal row. So "NGB2" already reaches the heavy-tailed one-sided family that a separate Lomax model would provide, and no such model_name is needed.

One caveat specific to the Lomax region. A Lomax with shape νu≤1\nu_u \le 1 has no finite mean, and with νu≤2\nu_u \le 2 no finite variance. A "NGB2" fit that settles near τ=ψu=1\tau = \psi_u = 1 with a small νu\nu_u is therefore in a region where E[u]E[u] does not exist, which is what exp_u_hat and the JLMS predictor are estimating. The fit itself is well defined – the likelihood only needs the density – but inefficiency summaries from it should not be read as estimates of a population mean that is not there.

νu\nu_u has no interior maximum once the tail is not heavy, and a large fitted value is not an estimate. Past a few hundred the density has stopped moving – the discrepancy from the generalized-gamma limit is 2e-4 at νu=106\nu_u = 10^6 and falls only as 1/νu1/\nu_u – so the likelihood is flat there. sfm() warns above 200; report that as “generalized gamma, no heavy tail detected”, treat the standard error on νu\nu_u as meaningless, and read ψu\psi_u and τ\tau, which still carry shape in that limit. Starting values are placed at the half-normal point of the family (τ=1\tau = 1, ψu=2\psi_u = 2, νu=30\nu_u = 30), so a fit that moves away from it has moved for a reason.

The composed density has no closed form and is evaluated by deterministic quadrature with two node sets, one at the quantiles of uu and one at the quantiles of the noise truncated to u>0u > 0, combined by the balance heuristic. Either alone fails: the first misses a narrow noise, the second a concentrated uu, and a four-parameter family can be either. Measured against adaptive integration over 243 parameter settings, at points where the density is not negligible (log⁡f>−20\log f > -20) the median error in log⁡f\log f is 1.2e-8 and the worst is 3.4e-2. The worst cases are one corner of the grid – σv=0.15\sigma_v = 0.15 against a spiky generalized gamma (ψu=8\psi_u = 8) – where adaptive integration disagrees with itself by up to 0.9, so it is not a reference that would justify more nodes. A single node set on uu's probability scale was wrong by up to 12 at the peak of the density on the same grid.

It is not estimated by simulated maximum likelihood, and the reason is worth knowing before reaching for simulation elsewhere. Written that way first, with 100 draws per proposal, the simulated likelihood was biased upward and the optimizer chased the bias: on n=400n = 400 generated from a half-normal it walked to ψu=88\psi_u = 88, τ=0.22\tau = 0.22 – a near point mass, where importance sampling from the inefficiency badly overstates the density – and reported a log-likelihood of -336.9 where the exact value at those parameters is -458.7. Simulated ML rewards whatever parameters maximise its own simulation error, and four shape parameters give it room to find them.

This is nonetheless the slowest sfm() model: about 0.08 s per likelihood evaluation at n=500n = 500, so a fit takes minutes rather than seconds.

"NB": a binomial inefficiency, and what wrong skewness might mean. Carree (2002). Every other one-sided distribution in this table is positively skewed, so a production frontier implies a negatively skewed composed error, and a positive residual skew has nowhere to go: the maximum likelihood estimate is σu=0\sigma_u = 0 (Waldman 1982), read as “no inefficiency”. Carree's observation is that this is not the only reading. A binomial b(n,p)b(n, p) is negatively skewed for p>1/2p > 1/2, and that says something quite different – most firms carry considerable inefficiency and only a few sit near the frontier. Same sample moment, opposite economics.

The estimator inverts the second, third and fourth central moments of the OLS residuals (his Eq. 7),

m2=σv2+np(1−p),m3=−np(1−p)(1−2p),m4−3m22=np(1−p)(1−6p+6p2),m_2 = \sigma_v^2 + np(1-p), \quad m_3 = -np(1-p)(1-2p), \quad m_4 - 3m_2^2 = np(1-p)(1-6p+6p^2),

whose ratio x=(m4−3m22)/m3=(6p2−6p+1)/(2p−1)x = (m_4 - 3m_2^2)/m_3 = (6p^2-6p+1)/(2p-1) is a quadratic in pp with roots p=12+x6±16x2+3p = \tfrac12 + \tfrac{x}{6} \pm \tfrac16\sqrt{x^2+3}. Which root applies is settled by xx outside [−1,1][-1,1] and by the sign of m3m_3 inside it. n_bin and p_bin are reported alongside sigu, which is np(1−p)\sqrt{np(1-p)} so that it is comparable with the other models.

Three things to know. First, a positive m3 is admissible here, which is exactly what it is for; wrong_skew is set only when the inversion genuinely fails. Second, it uses the fourth moment, so it is markedly more fragile than the half-normal and exponential inversions – as Greene's gamma estimator is, and for the same reason. Carree's infeasible region is ∣m4−3m22∣>∣m3∣>0|m_4 - 3m_2^2| > |m_3| > 0, where the implied nn is negative; sfm() warns, reports no estimate, and that should be read as evidence against a binomial inefficiency rather than as a number. It also needs σv\sigma_v not to be small relative to np(1−p)\sqrt{np(1-p)}, since σv2\sigma_v^2 is then a small difference of large moments. Third, the support is the integers: one unit of uu is one unit of the dependent variable, so with a log dependent variable each “inefficiency category” is a factor of ee. A scale factor is not identified from three moments alongside (n,p,σv)(n, p, \sigma_v), so there is none, and the implied efficiency levels should be read with that in mind. The efficiency predictor is the exact discrete posterior P(u=j∣ε)∝(nj)pj(1−p)n−jϕ(ε+j;0,σv)P(u = j \mid \varepsilon) \propto \binom{n}{j}p^j (1-p)^{n-j}\phi(\varepsilon + j; 0, \sigma_v), with nn rounded to an integer; n_bin keeps the unrounded moment estimate.

"TSL" assumes a truncated skew-Laplace inefficiency (Wang 2012, eq. 3.7; the distribution is due to Aryal and Rao 2005, and the model was published as Nguyen, Gupta and Wang 2014), with density f(u)=1+λσu(2λ+1)[2e−u/σu−e−(1+λ)u/σu]f(u) = \frac{1+\lambda}{\sigma_u(2\lambda+1)}\left[2e^{-u/\sigma_u} - e^{-(1+\lambda)u/\sigma_u}\right] for u≥0u \ge 0. This is a signed mixture of two exponentials – the second enters negatively – so it nests the exponential model as λ→0\lambda \to 0 while allowing a non-monotonic inefficiency density, and the composed density is the corresponding signed combination of two normal-exponential terms. It is evaluated as a difference in log space rather than by subtracting the raw terms: both carry σv2/(2σu2)\sigma_v^2/(2\sigma_u^2), which overflows once σu\sigma_u becomes small relative to σv\sigma_v, a region the optimizer routinely visits. lambda is a shape parameter and is the least sharply identified of the three. This is a property of the family, not of the fit, and it is worth understanding before reporting lambda at all. The shape parameter changes the standardised distribution remarkably little: E[U]/σuE[U]/\sigma_u equals 1 in both limits λ→0\lambda \to 0 and λ→∞\lambda \to \infty, and its maximum over all λ\lambda is only 4−22≈1.17164-2\sqrt{2} \approx \bold{1.1716}, attained at λ=1/2\lambda = 1/\sqrt{2}; the standardised skewness moves only from about 1.95 down to 1.79 and back. So the data identify the product σu×m(λ)\sigma_u \times m(\lambda) far better than they identify σu\sigma_u and λ\lambda separately, and the two trade off against each other.

Measured on the package's own convergence design (σu=1\sigma_u = 1, λ=1.5\lambda = 1.5, 60 replications at each of n=500,1000,2000,4000n = 500, 1000, 2000, 4000), estimating one parameter with the other held at its true value:

estimated log-MSE on log-n slope sd at n = 4000
σu\sigma_u, λ\lambda free -0.65 0.038
σu\sigma_u, λ\lambda fixed -1.12 0.020
λ\lambda, σu\sigma_u free -0.38 (median) 26.3
λ\lambda, σu\sigma_u fixed -1.07 (median) 0.62

Each parameter attains the root-nn rate when the other is known and fails it when the other is free, which is the signature of weak scale–shape separation rather than of an optimiser problem. λ\lambda's dispersion falls fortyfold once σu\sigma_u is fixed.

What to do about it. Report the functionals rather than the primitives where you can: E[U]E[U] converges at the root-nn rate even with both parameters free (slope -1.03, sd 0.031 at n=4000n = 4000), and E[e−U]E[e^{-U}] nearly so (-0.89). Both are what an efficiency analysis actually uses. If λ\lambda itself is of interest, prefer profile-based inference over its reported tt-value, which is a Wald statistic on a parameter whose sampling distribution is neither symmetric nor tightly concentrated.

Robust estimation (robust)

Robust divergence-based estimation (robust argument). For model_name = "NHN", sfm() can maximize a robust divergence objective instead of the ordinary log-likelihood, downweighting the influence of outlying observations:

  • "mlqe": minimum Lqq estimation (Lq, Ferrari and Yang 2010), power parameter c_mlqe (q=1−cq = 1-c). Objective ∑(f(ei)c−1)/c\sum (f(e_i)^c - 1)/c.

  • "psi": Psi-likelihood (Basu et al.-type divergence family), power parameter eta. Objective ∑f(ei)c/c−n I1+c/(1+c)\sum f(e_i)^c/c - n \, I_{1+c}/(1+c), where I1+c=∫f(e;θ)1+c deI_{1+c} = \int f(e;\theta)^{1+c}\,de is evaluated numerically at the current parameter values on every optimizer iteration.

  • "mdpd": minimum density power divergence (Basu et al. 1998), power parameter alpha. Objective ((1+c)/c)∑f(ei)c−n I1+c((1+c)/c) \sum f(e_i)^c - n \, I_{1+c}.

All three reduce to "mle" as their power parameter approaches 0. Psi and MDPD produce identical parameter point estimates for the same power parameter (c=η=αc = \eta = \alpha) – the two objectives are a positive rescaling of one another, MDPD(c)=(1+c)⋅Psi(c)\text{MDPD}(c) = (1+c)\cdot \text{Psi}(c), which does not change the argmax; both are exposed for compatibility with the corresponding paper's three-method terminology, not because they differ numerically. MLqE is a genuinely distinct estimator.

Because these are M-estimators rather than ordinary MLE, the naive Hessian-inverse standard error (optHessian = TRUE's usual output) is not statistically valid for them – it assumes the information-matrix equality, which these objectives do not satisfy in general. When robust != "mle", sfm() instead reports a sandwich-form standard error, A−1BA−1A^{-1} B A^{-1}, where AA is the Hessian of the summed robust objective (as already computed for the point estimate) and B=∑igigi′B = \sum_i g_i g_i' is built from the per-observation gradient of that objective (obtained via numerical differentiation, jacobian). This requires optHessian = TRUE; std.errors/t.values are NA otherwise, same as for ordinary MLE.

Formulas verified against an independently audited reference implementation (cross-checked by direct numerical integration and finite differences), including the sandwich SE's own verification (the c -> 0 limit against the ordinary MLE Hessian SE, and a Monte Carlo check against the empirical spread of repeated fits). Currently only implemented for model_name = "NHN" – other models will error clearly if robust != "mle" is requested.

Optimizer defaults

Note on default iteration counts: sfm()'s optimization defaults (thousands of iterations) are much higher than psfm()'s (100/10/10). This is intentional: sfm()'s likelihoods are closed-form (no simulation), so each objective-function evaluation is cheap and many default iterations cost little; several of psfm()'s models require simulated maximum likelihood over Halton draws, where each evaluation is far more expensive. See psfm's Details for more.

Value

An object of class "sfareg" containing the following components:

out

A matrix with parameter estimates, standard errors, and t-values.

opt

A list containing the optimization results from the final optimization procedure.

total_time

The total computation time for model estimation.

start_v

The starting values used in the optimization.

model_name

The name of the stochastic frontier model estimated.

formula

The formula used in the model specification.

exp_u_hat

Predicted technical efficiency, E[exp(-u)|e]. Available for models: NHN, NHN_Z, NR, NG, NNAK, NE, NTN, NU, NGE, NLN, NW, NGB2, THT and tHN, and for estimator = "cols" with NHN, NE, NG and NB. Each uses the posterior its own uu implies.

u_hat

Predicted inefficiency, E[u|e] (Jondrow et al. 1982). Available for NE, NTN, NU, NGE, NLN, NW, NGB2, THT and tHN.

n_bin, p_bin

model_name = "NB" only, in out and coefficients: the moment estimates of the binomial's size and success probability. n_bin is a real number, not an integer.

wrong_skew, sigma_u_at_bound, residual_m3

Wrong-skewness diagnostics. residual_m3 is the third central moment of the OLS residuals; wrong_skew is TRUE when that is non-negative, which for a production frontier means the moment equations admit no interior solution; sigma_u_at_bound is TRUE when the fitted one-sided scale has collapsed to the zero boundary. When both are TRUE the boundary is the maximum likelihood estimate – the Type I failure of Olson, Schmidt and Waldman (1980), studied by Waldman (1982) – and not a numerical failure, so sfm reports it rather than bounding σu\sigma_u away from zero. Read it as no evidence of inefficiency in these data, and treat the efficiency scores as uninformative. A collapse with correctly skewed residuals would be a different matter and does not occur in testing.

thn_sigma_u_at_bound

Logical, tHN only. TRUE when the fitted σu\sigma_u collapsed onto zero, so the model reports essentially no inefficiency. Accompanied by a warning. Treat the efficiency scores as uninformative when this is set.

ntn_sigma_v_at_bound

Logical, NTN only. TRUE when the fitted σv\sigma_v collapsed onto zero, i.e. lambda ran away, so the fit attributes all variation to inefficiency and none to noise. On such a sample the likelihood has no interior maximum in lambda – it rises monotonically toward a finite limit as lambda grows without bound – so the boundary is the maximum likelihood estimate rather than a numerical failure. The likelihood is flat in lambda there, so the Hessian is singular and every standard error is NA; no finite standard error for lambda exists at that point. Accompanied by a warning. Read it as "no noise component identified in these data" rather than as an estimate of lambda.

thn_starts

List, tHN only. Multi-start diagnostic: how many starting values were tried and converged, the log-likelihood reached from each, and n_distinct, the number of separate optima found. n_distinct > 1 means the likelihood surface is multimodal for these data and the single returned fit is only the best of those found.

estimator

"cols", "acols" or "cmle", naming the estimator that ran; absent for ordinary maximum-likelihood fits.

wrong_skew

Logical, estimator = "cols" and "acols" only. For "cols", TRUE when the OLS residuals were skewed the wrong way and the moment equations had no admissible solution. For "acols" the third moment is never consulted, so it is TRUE when the absolute-moment equation has no root on the admissible interval.

boundary

Logical, estimator = "cmle" only. TRUE when the two moment constraints could not both be met in the interior and one of σu\sigma_u, σv\sigma_v sits on its boundary.

residual_moments

The second, third and fourth central moments of the OLS residuals, estimator = "cols" only.

cols_boot_draws

The matrix of bootstrap replicates (cols_boot rows, one column per parameter) when cols_boot > 0; NULL otherwise.

ng_starts

Returned by NG only. A record of the starting-value search: how many candidates were built (n_tried), how many were polished before choosing (n_polished), the log-likelihood at each candidate (loglik_at_start), and the best value reached (best). NULL when start_val was supplied. The normal-gamma likelihood determines E[u]=μσuE[u] = \mu\sigma_u far better than it determines either factor – the weak identification Ritter and Simar (1997) describe – so the candidates hold E[u]E[u] at a moment estimate and sweep the shape along that ridge rather than searching across it. Inspect this if a normal-gamma fit looks implausible: widely differing log-likelihoods among the candidates indicate the surface is more awkward than usual for that data set.

sd_exp_u_hat

Standard deviation of exp(-u) given the residual, following Tancredi (2002, section 2.2). Returned by THT only. It is the quantity that distinguishes the skew-t frontier from the half-normal one: for a large positive residual the half-normal model concentrates exp(-u) at 1 and reports near-zero uncertainty, while the skew-t model treats the point as an outlier, widens this spread and does not push the efficiency estimate to 1.

med_u_hat

Predicted technical efficiency (median values). Available only for the NHN model.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE). Sandwich-form (not the ordinary Hessian-inverse form) when robust != "mle" – see Details.

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

call

The matched call.

het

Only present when vhet, uhet or muhet was given: the family fitted, the link used, the sizes of the four parameter blocks, the formulas as supplied, and (for "NTN") the fitted per-observation μ\mu.

sigma_u, sigma_v

Only present for a heteroskedastic fit: the fitted per-observation scales.

v_spec, mu_spec

Only present for a heteroskedastic fit: the noise and pre-truncation-mean blocks, in the same shape as z_spec.

robust

Only present for model_name = "NHN": the robust argument used to fit this model ("mle", "mlqe", "psi", or "mdpd").

robust_c

Only present for model_name = "NHN": the power parameter (c_mlqe/eta/alpha, whichever applies) used, or NA when robust = "mle".

Note

Standard errors require optHessian set to TRUE. When robust != "mle" (model_name = "NHN" only), a sandwich-form SE is reported instead of the ordinary Hessian-inverse one – see Details.

Author(s)

David H. Bernstein and Alexander Stead

References

Parmeter, C. F. and Zhao, S. (2023). An alternative corrected ordinary least squares estimator for the stochastic frontier model. Empirical Economics 64(6), 2831-2857. doi:10.1007/s00181-023-02401-1

Zhao, S. and Parmeter, C. F. (2022). The “wrong skewness” problem: Moment constrained maximum likelihood estimation of the stochastic frontier model. Economics Letters 221, 110901. doi:10.1016/j.econlet.2022.110901

Waldman, D. M. (1982). A stationary point for the stochastic frontier likelihood. Journal of Econometrics 18(2), 275-279. doi:10.1016/0304-4076(82)90041-0

Simar, L. and Wilson, P. W. (2009). Inferences from cross-sectional, stochastic frontier models. Econometric Reviews 29(1), 62-98. doi:10.1080/07474930903324523

See Also

sfa-wrongskew for the full wrong-skewness suite, psfm for panel models, zsfm for zero-inefficiency models, ttsfm for two-tier models, data_gen_cs for simulating data with known true parameters, and sfareg-methods for extracting results.

Examples

library(sfa)     

cs_data_trial   <- data_gen_cs(N= 1000, rand   = 1,  sig_u  = 0.3, sig_v  = 0.3, 
cons   = 0.5,       beta1  = 0.5,   beta2  = 0.5, a      = 4, mu     = 1)

cs.nhnz     <-  sfm(formula    = y_pcs_z ~ x1 +x2| z,    model_name = "NHN_Z",
                    data       = cs_data_trial,          PSopt      = TRUE)

## Robust divergence-based estimation (NHN only): downweights the influence
## of outlying observations relative to ordinary MLE. std.errors/t.values
## are NA for these methods -- see Details.
cs.nhn.mlqe <-  sfm(formula = y_pcs ~ x1 + x2, model_name = "NHN",
                    data = cs_data_trial, robust = "mlqe", c_mlqe = 0.20)
cs.nhn.psi  <-  sfm(formula = y_pcs ~ x1 + x2, model_name = "NHN",
                    data = cs_data_trial, robust = "psi", eta = 0.20)

## Heteroskedasticity in the noise as well as in the inefficiency: `| z`
## still drives sigma_u, `vhet` drives sigma_v. Coefficients come back
## prefixed by block (Zu., Zv.).
cs.het      <-  sfm(formula = y_pcs_z ~ x1 + x2 | z, model_name = "NHN_Z",
                    data = cs_data_trial, vhet = ~ z)

## Battese and Coelli (1995): covariates shift the LOCATION of inefficiency
## rather than its dispersion. Requires the truncated-normal family.
cs.bc95     <-  sfm(formula = y_pcs ~ x1 + x2, model_name = "NTN",
                    data = cs_data_trial, muhet = ~ z)

Average over inefficiency distributions instead of choosing one

Description

Fits several inefficiency distributions to the same data and combines their firm-level inefficiency predictions with data-driven weights. Where TIC and vuong answer "which distribution?", this answers what to do when the honest reply is "the data does not say" – selecting one specification and then reporting its efficiencies understates the uncertainty, because the standard errors condition on a choice that was itself estimated.

Usage

sfma(formula, data, models = c("NHN", "NE", "NTN", "NR", "NGE"),
     weights = c("sfma", "tic", "aic", "bic", "equal"),
     inefdec = TRUE, quiet = FALSE, ...)

Arguments

formula, data, inefdec, ...

Passed to sfm for each candidate.

models

Two or more model_names to average over. Candidates that fail to fit, or that report no E[u∣ε]E[u \mid \varepsilon], are dropped with a message.

weights

How to weight the candidates. See ‘Details’.

quiet

Suppress the messages about dropped candidates, including a candidate that gets no "tic" weight because its Takeuchi penalty is undefined (for instance with a parameter on its bound).

Details

The weighting schemes.

"sfma" is the criterion of Parmeter, Wan and Zhang (their eq. 13),

C(w)=∥ρ^(w)−ρ^full∥2+n1/2log⁡(n) k′w,C(w) = \|\hat\rho(w) - \hat\rho_{full}\|^2 + n^{1/2}\log(n)\,k'w,

minimised over the unit simplex. The unobservable target is replaced by the prediction from the most general candidate, exactly as Mallows' CpC_p replaces an unknown σ2\sigma^2 with the full model's estimate; without the penalty all weight would go to the largest candidate by construction. Their Theorem 1 says weights on models that nest the truth go to zero, and they do.

"tic", "aic" and "bic" are smoothed information-criterion weights in Buckland et al.'s (1997) form, ws∝exp⁡(−ICs/2)w_s \propto \exp(-IC_s/2). "tic" is the interesting one, because unlike AIC it does not assume the candidate is correctly specified – which is the premise of averaging in the first place. "equal" is the naive average, useful as a baseline.

"sfma" needs a unique most general candidate, and refuses without one. If every candidate has the same number of parameters the penalty n1/2log⁡(n)k′wn^{1/2}\log(n)k'w is identical across them, so it drops out as a constant and the criterion collapses to "be as close as possible to ρ^full\hat\rho_{full}" – minimised by putting weight 1 on whichever model was nominated as full. Measured on exponentially-distributed inefficiency with candidates "NHN", "NE" and "NR" (all with 5 parameters), that would have placed 0.879 on "NHN", the wrong distribution, where "tic" gave "NE" 0.934. Rather than break the tie silently, sfma errors and points at "tic". Including a genuinely more general candidate such as "NTN" makes the criterion well posed.

Reading the output. print reports the per-firm spread of efficiency across candidates. That spread is the reason to average: if the candidates agree, model choice was not costing anything and any single fit would have done; if they disagree, a single-model standard error is not describing the uncertainty a reader cares about.

Implementation. The weight problem is a convex quadratic program on the simplex, solved by projected gradient with an exact simplex projection (Duchi et al. 2008), so a weight that should be zero is exactly zero. It is checked against brute-force enumeration of the simplex in the tests rather than against itself.

Value

An object of class "sfma" with components weights, u_hat (the averaged E[u∣ε]E[u \mid \varepsilon]), efficiency, u_by_model (the n×Sn \times S matrix of per-candidate predictions), fits, k, logLik and, for weights = "sfma", criterion.

References

Buckland, S.T., Burnham, K.P. and Augustin, N.H. (1997). Model selection: an integral part of inference. Biometrics, 53(2), 603–618.

Duchi, J., Shalev-Shwartz, S., Singer, Y. and Chandra, T. (2008). Efficient projections onto the l1-ball for learning in high dimensions. ICML.

Huang, C.J. and Lai, H.-p. (2012). Estimation of stochastic frontier models based on multimodel inference. Journal of Productivity Analysis, 38(3), 273–284.

Parmeter, C.F., Wan, A.T.K. and Zhang, X. (2019). Model averaging estimators for the stochastic frontier model. Journal of Productivity Analysis, 51(1), 91–103.

See Also

TIC, vuong, spec_test, sfm

Examples

set.seed(3)
n  <- 300
x1 <- rnorm(n); x2 <- rnorm(n)
## True inefficiency is exponential.
d  <- data.frame(y = 1 + 0.5 * x1 + 0.5 * x2 +
                     rnorm(n, 0, 0.4) - rexp(n, 1),
                 x1 = x1, x2 = x2)

## Information-criterion weights: NE should earn most of the weight.
a <- sfma(y ~ x1 + x2, data = d, models = c("NHN", "NE", "NR"),
          weights = "tic", quiet = TRUE)
a
a$weights

## The Parmeter-Wan-Zhang criterion needs a uniquely largest candidate.
sfma(y ~ x1 + x2, data = d, models = c("NHN", "NE", "NTN"),
     weights = "sfma", quiet = TRUE)

Simulation Standard Errors for Simulated-ML Fits

Description

Reports how much of a fitted parameter's uncertainty is simulation noise rather than sampling noise, by refitting the same data with independent randomizations of the draw sequence.

Usage

simulation_se(object, K = 10, seeds = NULL, quiet = FALSE,
              envir = parent.frame())

Arguments

object

An "sfareg" fit from a simulated-ML model: sfm with "NLN" or "NW", psfm with "GTRE", "TRE", "TRE_Z" or "GTRE_Z", or selsfm.

K

Number of refits, at least 2.

seeds

Optional integer vector of randomization seeds; its length overrides K.

quiet

Suppress the per-refit progress messages.

envir

Environment in which to re-evaluate the stored call. The default is usually right; pass it explicitly if the data cannot otherwise be found.

Details

Simulated maximum likelihood replaces an integral with an average over draws. A standard error from the Hessian answers “how much would this estimate move in a new sample”, and says nothing about “how much would it move with the same sample and different draws”. Randomizing the sequence makes the second question answerable: refit KK times with independent shifts and take the standard deviation across refits.

This is possible only because the draws are randomized by a uniform shift. An unrandomized Halton lattice returns the same answer every time, so the diagnostic would report exactly zero and mean nothing by it.

Reading the result. ratio is the share of the reported standard error that comes from having simulated the integral. A large value is an argument for raising Nsim, not for collecting more data. On the package's reference design ("NW", n=400n = 400, Nsim = 200) the frontier slopes come back at about 0.5% while the variance and shape parameters are near 8% – which independently reproduces the documented behaviour that too few draws move everything except the slopes.

Quadrupling Nsim on that design cuts the simulation standard errors by roughly a factor of six, faster than the factor of two that independent pseudorandom draws would give. That is the low-discrepancy sequence earning its place.

Value

A data frame with one row per parameter: estimate, sampling_se (from the Hessian), simulation_se (across refits) and their ratio. Attributes n_refits and seed_arg record how many refits succeeded and which argument was varied.

See Also

sfa_diagnostics for convergence diagnostics on a single fit.

Examples

## Kept deliberately small: every refit is a full simulated-ML fit, so the
## cost is K + 1 of them. K = 3 is the smallest that still shows the spread.
d <- data_gen_cs(N = 200, rand = 1, sig_u = 1, sig_v = 0.3, cons = 0.5,
                 beta1 = 0.5, beta2 = 0.5, a = 1, mu = 0.5)
fit <- sfm(y_pcs_wb ~ x1 + x2, model_name = "NW", data = d, Nsim = 200)
simulation_se(fit, K = 3, quiet = TRUE)

Where the skewness of the composed error comes from

Description

Splits the third central moment of the composed error ε=v−u\varepsilon = v - u into the part contributed by the asymmetry of the inefficiency term, the part contributed by the asymmetry of the noise, and the part contributed by the dependence between them, following Bonanno, De Giovanni and Domma (2017).

Usage

skewness_decomp(object, nodes = 160L)

Arguments

object

A fitted model of class "sfareg": a copsfm fit, or an sfm fit with model_name = "NHN" or "NE". Other models error, because the decomposition needs the two error components to have separately identified third moments.

nodes

Number of Gauss-Legendre nodes per dimension for the cross moments and the median. The default is where the FGM case stops moving in the ninth significant digit; raise it if a copula with strong tail dependence is in use.

Details

Write u~=u−E[u]\tilde u = u - E[u] and v~=v−E[v]\tilde v = v - E[v]. Then

E[(ε−Eε)3]=E[v~3]−E[u~3]−3E[v~2u~]+3E[v~u~2],E[(\varepsilon - E\varepsilon)^3] = E[\tilde v^3] - E[\tilde u^3] - 3E[\tilde v^2 \tilde u] + 3E[\tilde v \tilde u^2],

and the last two terms vanish under independence. The function reports the three pieces as inefficiency (−E[u~3]-E[\tilde u^3]), noise (E[v~3]E[\tilde v^3]) and dependence (the two cross terms together).

The point of the decomposition is what it says about the textbook model. A normal vv sets the noise term to zero, and independence sets the dependence term to zero, so under the standard specification the sign of the composed third moment is the sign of −E[u~3]-E[\tilde u^3] and nothing else. Every one-sided inefficiency distribution in common use is positively skewed, so a production frontier forces a negative composed skew, and a positively skewed residual has nowhere to go but σ^u=0\hat\sigma_u = 0. Skew in the noise and dependence between the components are perfectly ordinary explanations for the same residual, and the textbook model cannot express either. That is the sense in which Bonanno, De Giovanni and Domma call the wrong-skewness anomaly an ill-posed problem.

Also reported is Zenga's median-based measure, their Eq. (13),

E[(ε−Me(ε))3]=E[(ε−Eε)3]+[Eε−Me(ε)]3+3[Eε−Me(ε)]Var(ε),E[(\varepsilon - Me(\varepsilon))^3] = E[(\varepsilon - E\varepsilon)^3] + [E\varepsilon - Me(\varepsilon)]^3 + 3[E\varepsilon - Me(\varepsilon)]\mathrm{Var}(\varepsilon),

whose sign is the one that says which way the density actually leans. When it disagrees in sign with the third central moment, print() says so: the mean-based measure is the misleading one in that case.

The marginal contributions are closed forms. The dependence term is computed by Gauss-Legendre quadrature on the unit square against the copula density, which covers every marginal and copula copsfm() supports rather than only the exponential/generalized-logistic/FGM combination for which the paper gives an explicit formula. The two agree to the quadrature's own accuracy. Note that the closed forms printed as Eq. (10) and Eq. (11) of the paper do not: their dependence terms disagree with a direct evaluation of the same expectations, and the corrected expressions – with the Gauss hypergeometric weights A1=−δv[ψ(2αv)−ψ(αv)]A_1 = -\delta_v[\psi(2\alpha_v) - \psi(\alpha_v)] and A2=δv2[ψ′(αv)−ψ′(2αv)−(ψ(2αv)−ψ(αv))2]A_2 = \delta_v^2[\psi'(\alpha_v) - \psi'(2\alpha_v) - (\psi(2\alpha_v)-\psi(\alpha_v))^2] – are the ones pinned in tests/testthat/test-wrongskew-suite.R.

Value

An object of class "sfa_skew_decomp", a list with components:

components

Named numeric of length three: inefficiency, noise, dependence. They sum to total.

total

The third central moment of the composed error implied by the fit.

share

components / total.

mean, median, variance

The first moment, median and variance of the composed error implied by the fit.

median_based

Zenga's measure, Eq. (13).

model_name, udist, vdist, copula, copula_par

What the decomposition was taken of.

References

Bonanno, G., De Giovanni, D. and Domma, F. (2017). The “wrong skewness” problem: a re-specification of stochastic frontiers. Journal of Productivity Analysis 47(1), 49-64. doi:10.1007/s11123-017-0492-8

Zenga, M. (1985). Statistica descrittiva. Turin: Giappichelli.

See Also

sfa-wrongskew for the other ways this package handles wrong skewness, copsfm for the model the decomposition was written for, skewness_test for a formal test on the residuals.

Examples

set.seed(21)
n <- 300
d <- data.frame(x1 = rnorm(n), x2 = rnorm(n))
d$y <- 0.5 + 0.8 * d$x1 - 0.4 * d$x2 + rnorm(n, 0, 0.5) - abs(rnorm(n, 0, 1))

## Under the textbook model two of the three components are identically zero.
f <- sfm(y ~ x1 + x2, data = d, model_name = "NHN", estimator = "cols")
skewness_decomp(f)


## With a skewed noise and a copula, all three are live.
g <- copsfm(y ~ x1 + x2, data = d, copula = "fgm",
            udist = "exponential", vdist = "glogistic", n_nodes = 64)
skewness_decomp(g)

Formal tests for wrong skewness in the OLS residuals

Description

Tests whether the OLS residuals of a stochastic frontier model are skewed in the direction the model implies. A production frontier has e=v−ue = v - u with u≥0u \ge 0, so the composed error is negatively skewed; residuals skewed the other way mean the maximum likelihood estimate of σu\sigma_u is exactly zero and every efficiency score is one.

Usage

skewness_test(object, test = c("agostino", "coelli"),
              alternative = c("auto", "less", "greater", "two.sided"))

Arguments

object

An object of class "sfareg" from sfm, or a numeric vector of residuals. From a fit, the OLS residuals recorded at fit time are used – not the composed residuals of the fitted model, which are not what either test is defined on.

test

"agostino" (the default) is D'Agostino's (1970) skewness test, which is the test Schmidt and Lin (1984) appeal to. "coelli" is Coelli's (1995) M3TM3T. See ‘Details’ for why the default is the former.

alternative

"auto" (the default) runs the one-sided test the model implies: a production frontier predicts negative skew, so the null of zero skew is tested against m3<0m_3 < 0. The other three force a direction.

Details

What the tests are. Both start from the third central moment of the OLS residuals. Coelli's M3T=m3/6m23/nM3T = m_3/\sqrt{6m_2^3/n} is asymptotically standard normal under the null of zero skewness. D'Agostino's transforms b1=m3/m23/2\sqrt{b_1} = m_3/m_2^{3/2} to approximate normality in finite samples rather than leaning on the asymptotic variance 6/n6/n.

Why "agostino" is the default. It holds its size where the asymptotic form does not. Measured here, one-sided at a nominal 5% over 4,000 replications of symmetric residuals:

nn 25 50 100 400
coelli 0.031 0.037 0.043 0.050
agostino 0.045 0.046 0.046 0.051

Coelli's is conservative below about n=100n = 100; the two agree by n=400n = 400. The default also matches sfaR's. The implementation agrees with moments::agostino.test to within 10−1010^{-10}.

How to read the result. A small p-value under the default alternative is evidence for the frontier specification: the residuals are skewed the way inefficiency makes them. A large p-value is not evidence of a problem by itself – it may simply mean there is little inefficiency to detect, or too little data to detect it. What matters is the combination with wrong_skew: residuals skewed the wrong way and a fit reporting $sigma_u_at_bound is the Type I failure of Olson, Schmidt and Waldman (1980), in which case the boundary really is the maximum likelihood estimate (Waldman 1982) and the efficiency scores carry no information. See sfm.

Value

An object of class "htest", so it prints like any other R test. Alongside the usual statistic, p.value and estimate (the sample skewness b1\sqrt{b_1}) it carries m3, nobs, and wrong_skew – the last being TRUE when the third central moment is non-negative, matching sfm's own $wrong_skew.

References

Coelli, T. (1995). Estimators and hypothesis tests for a stochastic frontier function: a Monte Carlo analysis. Journal of Productivity Analysis, 6(3), 247–268.

D'Agostino, R.B. (1970). Transformation to normality of the null distribution of g1g_1. Biometrika, 57(3), 679–681.

Olson, J.A., Schmidt, P. and Waldman, D.M. (1980). A Monte Carlo study of estimators of stochastic frontier production functions. Journal of Econometrics, 13(1), 67–82.

Schmidt, P. and Lin, T.-F. (1984). Simple tests of alternative specifications in stochastic frontier models. Journal of Econometrics, 24(3), 349–361.

Waldman, D.M. (1982). A stationary point for the stochastic frontier likelihood. Journal of Econometrics, 18(2), 275–279.

See Also

moment_range and spec_test for the Type II counterpart, sfm, sfa_diagnostics

Examples

cs <- data_gen_cs(N = 200, rand = 1, sig_u = 0.3, sig_v = 0.4, cons = 0.5,
                  beta1 = 0.5, beta2 = 0.5, a = 5, mu = 0.1)
fit <- sfm(y_pcs_e ~ x1 + x2, model_name = "NE", data = cs)

skewness_test(fit)                    # D'Agostino, one-sided
skewness_test(fit, test = "coelli")   # Coelli's M3T

## Also works on a bare residual vector.
skewness_test(residuals(lm(y_pcs_e ~ x1 + x2, data = cs)))

Is this pair of distributions defensible?

Description

Tests an assumed pair of distributions for the composed error ε=v±u\varepsilon = v \pm u using only the skewness and excess kurtosis of the OLS residuals. It needs no frontier fit, so it can be run before estimating anything and cannot be contaminated by an optimiser that failed. spec_test_all runs all twelve supported pairs and ranks them.

Usage

spec_test(object, noise = c("normal", "logistic", "laplace", "uniform"),
          inefficiency = c("halfnormal", "exponential", "genexponential"),
          null = c("bootstrap", "asymptotic"),
          B = 999, level = 0.05, seed = NULL)

spec_test_all(object, level = 0.05)

Arguments

object

An "sfareg" fit, whose OLS residuals are used, or a numeric vector of residuals.

noise

Assumed distribution of vv. Only its excess kurtosis enters, so any symmetric distribution with the same value behaves identically – testing for Logistic noise is indistinguishable from testing for t(9)t(9).

inefficiency

Assumed distribution of uu.

null

How to get the p-value. "bootstrap" (the default) simulates from the assumed pair; "asymptotic" uses the delta-method normal approximation, which is badly over-sized for heavy-tailed uu – see ‘Details’.

B

Bootstrap replications. Cheap here: nothing is fitted, so each replicate is nn draws and eight moments.

level

Size used for the reported verdict.

seed

Optional seed for the bootstrap. The RNG state is saved and restored.

Details

What it tests. Write R=SNR2/(1+SNR2)R = \mathrm{SNR}^2/(1+\mathrm{SNR}^2) for the share of composed-error variance contributed by uu. Papadopoulos and Parmeter (2021, eqs. 5 and 10) give

∣γ1(ε)∣=γ1(u) R3/2,γ2(ε)=γ2(v)(1−R)2+γ2(u)R2.|\gamma_1(\varepsilon)| = \gamma_1(u)\,R^{3/2}, \qquad \gamma_2(\varepsilon) = \gamma_2(v)(1-R)^2 + \gamma_2(u)R^2.

With both distributions assumed, γ1(u)\gamma_1(u), γ2(u)\gamma_2(u) and γ2(v)\gamma_2(v) are known constants, so the skewness alone identifies RR and the kurtosis equation has nothing left to fit. The statistic is the gap,

Γ=γ2(ε)−γ2(v)(1−R)2−γ2(u)R2,\Gamma = \gamma_2(\varepsilon) - \gamma_2(v)(1-R)^2 - \gamma_2(u)R^2,

which is zero when the assumed pair is right. Its variance comes from the delta method applied to the central moments (m2,m3,m4)(m_2, m_3, m_4), which needs moments up to the eighth.

Why the default null is a bootstrap. The delta-method p-value is not reliable at usual sample sizes, and the failure is systematic rather than marginal. Rejection rates under correct specification, 300 replications per cell:

uu nn SNR asymptotic bootstrap
half-normal 2000 1 0.087 0.050
half-normal 2000 2 0.103 0.057
exponential 2000 1 0.207 0.077
exponential 500 2 0.357 0.083
exponential 2000 2 0.307 0.060

Across all 24 cells the asymptotic null rejects 18.7% of the time at a nominal 5%; the bootstrap rejects 5.7%. The cause is the eighth moment: for a heavy-tailed uu it is estimated far too imprecisely at these sample sizes, and the variance comes out too small. The bootstrap studentises against the same statistic computed under the assumed pair, so it corrects the variance rather than merely relocating the statistic. Power is unaffected – against wrong pairs at n=2000n = 2000 it runs 0.93 to 1.00.

Type II failure. If R>1R > 1 the residuals are more skewed than the assumed uu can be – for instance ∣γ1∣>0.9953|\gamma_1| > 0.9953 under a half-normal. The pair is then refuted by the skewness alone, and spec_test says so instead of reporting a p-value that would suggest the question was close.

Reading spec_test_all. These are not sequential tests: no one of them depends on another's outcome, so the procedure implies no multiplicity correction. But picking the largest of twelve p-values is a selection, and the ranking should be read as comparative evidence rather than as a significance level.

Relation to skewness_test. That asks whether the residuals are skewed the way inefficiency makes them – a test of whether there is inefficiency. This asks whether the assumed distributions are the right ones. They answer different questions and are worth running together.

Provenance. The statistic here is reconstructed from PP2021 equations (5) and (10); the Supplementary Appendix in which PP2023 derive it was not available. The reconstruction is verified two ways in the tests: Γ\Gamma vanishes under correct specification across sixteen designs, and the tabulated constants cu,1c_{u,1} and cu,2c_{u,2} are reproduced exactly as γ1(u)−2/3\gamma_1(u)^{-2/3} and 1/γ2(u)1/\gamma_2(u). PP2021's own equation (13) is deliberately not used, since PP2023 record that its HkH_k terms are missing squares.

Value

spec_test returns an object of class c("sfa_spec_test", "htest"), carrying statistic (the studentised Γ\Gamma), p.value, Gamma, se, R, skewness, excess_kurtosis, type2_failure and reject. spec_test_all returns a data frame of all twelve pairs sorted by p-value.

References

Papadopoulos, A. and Parmeter, C.F. (2021). Type II failure and specification testing in the Stochastic Frontier Model. European Journal of Operational Research, 293(3), 990–1001.

Papadopoulos, A. and Parmeter, C.F. (2023). A specification test for the composed error term in the stochastic frontier model. Economics Letters, 233, 111390.

See Also

moment_range for the attainable ranges the same two equations imply, skewness_test, TIC, sfm

Examples

set.seed(2)
n <- 2000
e <- rnorm(n, 0, 0.6) - abs(rnorm(n, 0, 1))   # normal / half-normal

## The true pair is not rejected.
spec_test(e, noise = "normal", inefficiency = "halfnormal",
          B = 199, seed = 1)

## A wrong one is.
spec_test(e, noise = "laplace", inefficiency = "exponential",
          B = 199, seed = 1)

## All twelve, ranked.
spec_test_all(e)

Summary method for "sfareg" objects

Description

Summary of a stochastic frontier fit, as returned by sfm, psfm, zsfm or ttsfm.

Usage

## S3 method for class 'sfareg'
summary(object, ...)

Arguments

object

An object of class "sfareg", as returned by sfm, psfm, zsfm or ttsfm.

...

Additional arguments passed to other methods

Details

Prints the call, the model name, and the estimated parameters with standard errors and t-values, together with the log-likelihood and the number of observations where the estimator provides them. Which parameters appear depends on the model: several report the λ=σu/σv\lambda = \sigma_u/\sigma_v, σ=σu2+σv2\sigma = \sqrt{\sigma_u^2 + \sigma_v^2} reparameterization rather than the raw scale parameters.

Value

Invisibly returns its object argument unchanged, having printed the summary as a side effect.

Author(s)

David H. Bernstein

See Also

print.sfareg, sfareg-methods

Examples

library(sfa)     

cs_data_trial   <- data_gen_cs(N= 1000, rand   = 1,  sig_u  = 0.3, sig_v  = 0.3, 
cons   = 0.5,       beta1  = 0.5,   beta2  = 0.5, a      = 4, mu     = 1)

cs.nhnz     <-  sfm(formula    = y_pcs_z ~ x1 +x2| z,    model_name = "NHN_Z",
                    data       = cs_data_trial,          PSopt      = TRUE)
summary(cs.nhnz)

Choosing between non-nested stochastic frontier specifications

Description

Model selection for stochastic frontier models when the candidates are not nested and none of them need be correctly specified. TIC is Takeuchi's information criterion, a version of AIC whose penalty is estimated from the data rather than assumed equal to the number of parameters. vuong is Vuong's pairwise test, which returns a p-value and is allowed to conclude that two specifications are indistinguishable.

Usage

TIC(object, detail = FALSE)

vuong(object1, object2, correction = c("none", "aic", "tic"), level = 0.05)

Arguments

object, object1, object2

Objects of class "sfareg", fitted with keep_objective = TRUE (see ‘Details’).

detail

If FALSE (the default) TIC returns a single number. If TRUE it returns a list also holding the log-likelihood, the parameter count, the estimated penalty and their ratio.

correction

Bias correction applied to the log-likelihood difference before it is standardised. "none" (the default) is the statistic whose N(0,1)N(0,1) limit Vuong established. "aic" subtracts the parameter counts and "tic" the Takeuchi penalties; both shift the mean of the statistic and neither has an established null distribution, so their p-values are flagged as indicative.

level

Two-sided size used for the reported critical value and the verdict.

Details

Why not AIC. sfa offers many model_names for the same data, and AIC assumes the fitted model is correctly specified – exactly the assumption in doubt when choosing between "NHN", "NE", "NG" and "NTN". Most such pairs are not nested either, so the ordinary likelihood ratio test has no chi-square limit.

What TIC does. Takeuchi's criterion replaces AIC's penalty dd with tr[H(θ^)I(θ^)−1]\mathrm{tr}[H(\hat\theta) I(\hat\theta)^{-1}], where II is the sample Fisher information and HH the outer product of the per-observation scores. Under correct specification the information matrix equality gives H=IH = I, the trace collapses to dd, and TIC equals AIC. The gap between penalty and df – reported as ratio under detail = TRUE – is therefore a readable diagnostic in its own right: a ratio near one says the distributional assumption is not doing visible damage. Both penalty and df are counted on the scale the model is ESTIMATED on, which for ivsfm("IVLIML") is not the scale it reports: it maximises over 11 parameters and reports 6, and comparing the trace against the reported count made the ratio read near 2 when it is near 1.

What vuong does. It tests H0H_0 that the two models are equally close to the truth in the Kullback-Leibler sense. Neither model has to be correct. With mim_i the difference of per-observation log-likelihoods, the statistic is n−1/2∑imi/σ^n^{-1/2}\sum_i m_i / \hat\sigma, asymptotically standard normal. Because it is a test rather than a criterion it can return "neither", which is frequently the honest answer and is not available from AIC.

Requirement. Both need per-observation log-likelihoods, so the fits must be made with keep_objective = TRUE. This is supported by sfm (for "NHN", "NE", "NR", "NG", "NNAK", "THT", "NTN", "NHN_Z", "NE_Z", "NU", "NGE", "NLN", "NW", "tHN" and "TSL"), by psfm for "GTRE" (estimator = "sml"), "TRE", "GTRE_Z", "TRE_Z", "GTRE_FML", "TFE_WMLE", "FD", "PL80", "PL80_MVTN", "BC92", "K1990" and "K1990modified" (from 1.2.1), whose contributions are per firm rather than per observation, and from 1.2.1 by zsfm, lcsfm, ttsfm ("TTNE", "TTHN"), copsfm, ivsfm ("IVLIML", "IVCF") and selsfm. vuong additionally requires both fits to use the same observations, which it checks by sample size and warns about when the data arguments differ.

Estimators that maximise nothing. psfm's "GTRE_SEQ1", "GTRE_SEQ2" and "SSFE", and ivsfm's "C2SLS", are moment-based or FE and have no log-likelihood, so no likelihood-based criterion applies to them and both functions refuse rather than returning a number built on NA.

Value

TIC returns a single numeric value, or with detail = TRUE a list with components TIC, AIC, logLik, df, penalty and ratio.

vuong returns an object of class "sfa_vuong" with components statistic, p.value, n, lr, lr_adjusted, omega, correction, penalty, critical, level, models, favoured and exact_null. Lower TIC is better; favoured is the name of the winning model_name, or "neither".

References

Lai, H.-p. and Huang, C.J. (2010). Likelihood ratio tests for model selection of stochastic frontier models. Journal of Productivity Analysis, 34(1), 3–13.

Takeuchi, K. (1976). Distribution of informational statistics and a criterion of model fitting. Suri-Kagaku (Mathematical Sciences), 153, 12–18.

Vuong, Q.H. (1989). Likelihood ratio tests for model selection and non-nested hypotheses. Econometrica, 57(2), 307–333.

See Also

sfm, skewness_test, sfa_diagnostics, AIC

Examples

set.seed(42)
n  <- 400
x1 <- rnorm(n); x2 <- rnorm(n)
d  <- data.frame(y = 1 + 0.5 * x1 + 0.5 * x2 +
                     rnorm(n, 0, 0.4) - abs(rnorm(n)),
                 x1 = x1, x2 = x2)

a <- sfm(y ~ x1 + x2, model_name = "NHN", data = d, keep_objective = TRUE)
b <- sfm(y ~ x1 + x2, model_name = "NE",  data = d, keep_objective = TRUE)

## Lower is better, and comparable across non-nested models.
TIC(a)
TIC(b)

## Is the penalty near the parameter count? If so, TIC has little to add.
TIC(a, detail = TRUE)$ratio

## And a test, which may decline to pick a winner.
vuong(a, b)

Is a collapsed two-tier noise scale the maximum, or did the optimizer stop short?

Description

Refits a two-tier frontier from several starting values of σv\sigma_v, scores every solution on the same likelihood, and reports which is highest. Use it when ttsfm returns σ^v≈0\hat\sigma_v \approx 0 and you need to know whether that is the maximum likelihood estimate or a failure of the optimizer.

Usage

tt_boundary_report(object, data, sigma_v_starts = c(0.05, 0.3, 1),
                   tol = 1e-3, tol_loglik = 1e-4, ...)

Arguments

object

A ttsfm fit with model_name = "TTHN" or "TTNE".

data

The data the model was fitted to. ttsfm() does not retain it and every refit here needs it.

sigma_v_starts

Positive starting values for σv\sigma_v. One refit is run from each, with every other parameter started at the fitted values.

tol

Below this, the returned σv\sigma_v counts as collapsed.

tol_loglik

How much higher a restart's log-likelihood has to be before it counts as higher at all. Restarts landing on the same optimum differ in the last few digits; without this they would be reported as an improvement of zero log units.

...

Passed to ttsfm for each refit.

Details

This is a likelihood comparison and not a test. No p-value is reported and none should be inferred. The hypothesis a reader would naturally attach to it – σv=0\sigma_v = 0 – sits on the boundary of the parameter space, where the usual chi-square reference distribution for a likelihood-ratio statistic does not apply, and no appropriate null distribution has been established for this model. The output is a table of log-likelihoods and a statement about which is largest.

Why it is needed. "TTHN" drives σv\sigma_v to zero when σu\sigma_u and σw\sigma_w are close, and that collapse is the maximum likelihood estimate rather than a defect: with σu=σw\sigma_u = \sigma_w the difference w−uw - u is already symmetric and heavier-tailed than a normal, so a small normal vv underneath it is very nearly invisible. On a 2,000-observation draw at σu=σw=1\sigma_u = \sigma_w = 1 the profile in σv\sigma_v is flat to 0.000 log units from 10−610^{-6} upward and decreasing above 0.01, with no interior maximum. Distinguishing that from an optimizer that merely stopped early is exactly what this function does.

The ordinary normal linear model on the same data is reported alongside, because a collapsed σv\sigma_v is not the same claim as “there is no frontier here”: on the design above the collapsed two-tier fit still beats the linear model by 9.6 log units, with both tiers estimated at 1.081 against truths of 1.0.

Cost. One ttsfm() refit per entry in sigma_v_starts. That is inexpensive for "TTNE" and expensive for "TTHN", where a single fit at 2,000 observations can take several minutes.

Value

An object of class "sfa_tt_boundary", a list with components:

table

A data frame with one row per start: the start, the σv\sigma_v it reached, and the log-likelihood.

best

Which row attained the highest log-likelihood. Ties go to the returned fit.

returned_is_best

TRUE when no restart beat the fit as returned by more than tol_loglik.

gap

The best restart's log-likelihood less the returned fit's.

collapsed

TRUE when the returned σv\sigma_v is below tol.

lm_logLik, lm_gap

The ordinary linear model's log-likelihood, and how far the fit beats it.

References

Polachek, S. and Yoon, B. J. (1987). A two-tiered earnings frontier estimation of employer and employee information in the labor market. The Review of Economics and Statistics 69(2), 296-302. doi:10.2307/1927239

See Also

ttsfm, and sfa_diagnostics for optimizer diagnostics generally.

Examples

d <- data_gen_cs(N = 1200, rand = 5, sig_u = 1, sig_v = 0.3, cons = 0.5,
                 beta1 = 0.5, beta2 = 0.5, a = 5, mu = 0, sig_w = 0.5)
f <- ttsfm(y_ttne ~ x1 + x2, data = d, model_name = "TTNE")
tt_boundary_report(f, data = d, sigma_v_starts = c(0.05, 0.8))

Two-Tier Stochastic Frontier Model

Description

Fits the two-tier stochastic frontier model (2TSF): the composed error is a symmetric noise term plus two one-sided components pulling in opposite directions, ε=v−u+w\varepsilon = v - u + w (for example buyer and seller informational rents).

  • "TTNE": uu and ww exponential.

  • "TTHN": uu and ww half-normal.

  • "TTNLS": nonlinear least squares, with no distributional assumption. It estimates the frontier slopes only; see ‘Details’.

Usage

ttsfm(formula, model_name = c("TTNE", "TTHN", "TTNLS"),
data, z_link = c("sd", "var"), maxit.bobyqa = 80000,
maxit.psoptim = 1000, maxit.optim = 1000,
REPORT = 1, trace = 0, pgtol = 0, start_val = FALSE, PSopt = FALSE,
optHessian = TRUE, inefdec = TRUE, upper = NA,
Method = "L-BFGS-B", logit = TRUE, verbose = FALSE,
keep_objective = FALSE, rand.psoptim = NULL)

Arguments

formula

A symbolic description of the frontier, y ~ x1 + x2. Use y ~ x | zu | zw to allow determinants zu of the first one-sided component and zw of the second.

model_name

"TTNE", "TTHN" or "TTNLS"; see ‘Description’. Matching ignores case, so "ttne" and "TTNE" are the same choice.

data

A data frame containing the variables named in formula.

z_link

Scale on which the variance-determinant linear predictors are placed. "sd" (the default, and ttsfm's historical convention) sets σu=exp⁡(zu′δ)\sigma_u = \exp(z_u'\delta) and likewise for σw\sigma_w; "var" sets σu=exp⁡(zu′δ)\sigma_u = \sqrt{\exp(z_u'\delta)}, matching psfm and the competing packages. The two are reparameterizations of the same model – identical maximised log-likelihood and identical frontier coefficients – but δ\delta under the SD link is exactly half δ\delta under the variance link, so a δ\delta from ttsfm is not comparable with one from psfm or sfaR unless the same convention is used. See sfm, which gained the same argument.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine.

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine.

maxit.optim

Maximum number of iterations for the optim optimization routine.

REPORT, trace, pgtol, upper

Accepted but not used by ttsfm(): the optimizer's own tracing is controlled by verbose, and its bounds are set internally.

start_val

FALSE (the default) to build starting values from the data, or a numeric vector giving them directly, in the optimizer's parameter order.

PSopt

Logical. Run the particle-swarm stage (pso::psoptim) between bobyqa and optim. FALSE by default. The swarm is the only stochastic step in ttsfm(): with the default, a fit is fully deterministic given the data and does not depend on the session's random-number state. See rand.psoptim.

optHessian

Logical. Finish with optim() and a numerically differentiated Hessian? Standard errors and t-values require TRUE, the default.

inefdec

Has no effect in ttsfm(): the composed error is v−u+wv - u + w whichever value is given, and TRUE and FALSE return identical fits. Read sigma_u as the component that lowers yy and sigma_w as the one that raises it.

Method

Method passed to optim for the final stage. Must be "L-BFGS-B": it is the only optim method that applies the lower and upper bounds this package uses to keep scale parameters positive, and any other value is refused with an error rather than quietly optimized without them.

logit

Accepted but not used by ttsfm().

verbose

Logical. Print optimization progress messages? Default is FALSE.

keep_objective

Logical. Retain the likelihood closure on the fitted object as $objective, so the score-based tools can difference it: vcov(type = "bhhh") (OPG/BHHH standard errors), influence_sfa, the sandwich methods, TIC and vuong. New in 1.2.1, for "TTNE" and "TTHN". It has no effect for "TTNLS", which is nonlinear least squares rather than maximum likelihood and so has no score; that case warns. FALSE by default, because a closure carries its enclosing environment.

rand.psoptim

Integer seed for the particle-swarm stage, used only when PSopt = TRUE. The seed is applied to that stage alone and the caller's random-number state is restored afterwards. With PSopt = TRUE and the default NULL the swarm draws from the session's random-number state, so set a seed whenever a result must be reproducible across sessions or package versions.

Details

The model

The two-tier stochastic frontier model (Polachek and Yoon, 1987) writes the composed error as ε=v−u+w\varepsilon = v - u + w, with vv symmetric normal noise and uu, ww independent one-sided components: uu lowers yy below the frontier and ww raises it above. "TTNE" assumes uu and ww are exponential; "TTHN" assumes they are half-normal.

Determinants of uu and ww enter through the second and third parts of a two-pipe formula, y ~ x | zu | zw, which parameterize σu\sigma_u and σw\sigma_w through a log link (see z_link).

"TTNLS"

Nonlinear least squares makes no distributional assumption, and the price is that it identifies only the frontier slopes and a composite intercept. E[y∣x]=x′β−E[u]+E[w]E[y \mid x] = x'\beta - E[u] + E[w], so the sum of squares cannot separate σu\sigma_u, σw\sigma_w and the intercept. ttsfm() therefore:

  • reports an intercept that absorbs E[w]−E[u]E[w] - E[u], with standard errors for the intercept and slopes from the least-squares covariance s2(X′X)−1s^2 (X'X)^{-1}. Without determinants TTNLS is ordinary least squares, and its coefficients and standard errors equal those of lm(); with determinants the standard errors are conditional on the fitted determinant coefficients;

  • returns NA for sigma_v, sigma_u and sigma_w, and for any determinant coefficients, with a warning;

  • returns metrics whose entries are all NA, since they are functions of those scales.

Use "TTNE" or "TTHN" when the individual scales are needed.

Boundary fits and metrics

Like the other composed-error models in this package, "TTNE" and "TTHN" can have a genuine (non-error) boundary mode where a variance component is estimated near its numerical floor in some samples; spot-check fits before serious use. "TTHN"'s information-deficiency metrics (the metrics component) involve a large number of bivariate-normal-CDF evaluations and are more experimental than the parameter estimates themselves; they are NULL if that calculation fails.

Iteration defaults

Like sfm() and zsfm(), ttsfm()'s likelihoods are closed-form, so its optimization defaults are much higher than psfm()'s, which uses simulated maximum likelihood for several models. See psfm's Details.

Value

An object of class "sfareg" containing the following components:

out

A matrix with parameter estimates, standard errors, and t-values, one row per parameter.

Scale parameters are reported on their natural scale and named, as of 1.2.1. The rows are sigma_v, sigma_u and sigma_w. Before 1.2.1 this matrix held raw optimizer values – the row labelled sigv was log⁡σv\log\sigma_v, and in the homoskedastic case the two one-sided scales were both labelled (Intercept), so uu and ww could not be told apart. Code that exponentiated these rows must stop doing so.

Where σu\sigma_u or σw\sigma_w carries determinants there is no single scale to report, so those rows stay as coefficients on the link and take the prefixes Zu. and Zw. – which is also what makes the two blocks distinguishable when they share a covariate name. sigma_v is homoskedastic in every case and is always reported directly.

A t-value on a scale parameter is not a conventional test. The null it corresponds to sits on the boundary of the parameter space, where the usual normal reference distribution does not apply. The column is reported for consistency with the rest of the package; read it as a rough magnitude, not as a p-value.

opt

A list containing the optimization results from the final optimization procedure.

total_time

The total computation time for model estimation.

start_v

The starting values used in the optimization.

model_name

"TTNE", "TTHN" or "TTNLS".

formula

The formula used in the model specification.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE).

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

metrics

A list of information-deficiency measures (M1, M2, M5, M6, M7, M10; see Polachek and Yoon) evaluated at the final parameter estimates. NULL if the underlying calculation failed, and all NA for "TTNLS"; see ‘Details’.

call

The matched call.

nobs

The number of observations used in the fit.

objective

The likelihood closure, when keep_objective = TRUE ("TTNE" and "TTHN" only).

Note

Standard errors require optHessian = TRUE.

"TTHN" will drive σv\sigma_v to zero when σu\sigma_u and σw\sigma_w are close, and that is the maximum likelihood estimate rather than a failure of the optimizer. With σu=σw\sigma_u = \sigma_w the difference w−uw - u is already symmetric and heavy-tailed, so a small normal vv underneath it is very nearly invisible: on a 2,000-observation draw at σu=σw=1\sigma_u = \sigma_w = 1 the σv\sigma_v profile is flat to 0.000 log units from 10−610^{-6} upward and decreasing above 0.01, with no interior maximum, and the collapsed fit beats the true parameter values by 1.29 log units. The two-tier structure is identified; the noise underneath two equal and opposite tiers is not. Asymmetric (σu,σw)(\sigma_u, \sigma_w) does not show this.

Author(s)

David H. Bernstein and Christopher F. Parmeter

References

Polachek, S. and Yoon, B.J. (1987). A Two-Tiered Earnings Frontier Estimation of Employer and Employee Information in the Labor Market. The Review of Economics and Statistics.

Examples

library(sfa)

data <- data_gen_cs(N = 200, rand = 1, sig_u = 0.3, sig_v = 0.2,
                     cons = 2, beta1 = 0.5, beta2 = 0.3, a = 5, mu = 0)

eq <- y_pcs ~ x1 + x2

ttsfm(formula    = eq,
      model_name = "TTNE",
      data       = data)

Test whether technical inefficiency depends on firm characteristics

Description

The two-step test of Kim and Schmidt (2008): regress the JLMS predictor on firm characteristics and test the slopes, with the variance correction that makes the test valid.

Usage

uhet_test(object, z, data = NULL)

Arguments

object

An "sfareg" fit from sfm with model_name = "NHN" or "NE", fitted without uhet and with keep_objective = TRUE. This is the first step, estimated under the null.

z

A one-sided formula (e.g. ~ z1 + z2) or a numeric matrix of the characteristics whose effect on inefficiency is being tested.

data

The data the model was fitted to.

Details

The procedure applied research actually runs is: fit the frontier, take u^i=E[ui∣εi]\hat u_i = E[u_i \mid \varepsilon_i], regress it on ziz_i, and test the slopes. As usually performed that test is invalid. u^\hat u is a generated dependent variable, so the first-step estimation error in ψ\psi enters the second-step variance; the naive test omits it. Kim and Schmidt show the omitted term is GriG r_i with G=E[zi∇ψbi]G = E[z_i \nabla_\psi b_i], so the naive test is valid only when G=0G = 0 – which holds when zz is independent of xx, and generally fails when they are correlated, the case applied work is usually in. There is no distribution-free correction, which is why only the two cases the paper works through are implemented.

What the correction buys. With zz independent of xx the two tests agree, as the theory says they must. With zz strongly correlated with xx (ρ=0.9\rho = 0.9) the naive standard error is 52% too large at n=800n = 800 and the naive test's size collapses to 0.000 at a nominal 5%, while the corrected standard error matches the true sampling spread to within half a percent (ratio 0.995) and the test retains size 0.032. A test that never rejects also has no power against local alternatives.

An honest caveat on size. The corrected test is conservative in these simulations: on Kim and Schmidt's own base case (scaled exponential, σv2=λ=1\sigma_v^2 = \lambda = 1, ρ=0.5\rho = 0.5, 1000 replications) it rejects 0.024, 0.036 and 0.028 of the time at N=200,500,1000N = 200, 500, 1000 against a nominal 0.05, where the paper reports 0.0495 for its corrected two-step test at N=200N = 200. The naive branch reproduces the paper's figure for the uncorrected test closely (0.023–0.029 against their 0.0237). The implementation was checked line by line against equations (5)–(13), the first-step scores agree with an analytic half-normal score to 10−1110^{-11}, γ^\hat\gamma is unbiased and Gaussian (kurtosis 2.92), and an oracle test using the true sampling standard deviation has correct size (0.045) – so the discrepancy is in the variance estimate, not in γ^\hat\gamma or the reference distribution: cor(∣γ^∣,se)cor(|\hat\gamma|, se) is 0.375, which compresses the ratio to a standard deviation of 0.93.

Two candidate explanations were tested and both refuted. Replacing the robust second-step variance with the homoskedastic one – the difference between the paper's “OLS” and “OLS-H” variants – changes nothing (0.023/0.027/0.036 against 0.022/0.036/0.031). Nor does using the Hessian information in place of the outer-product one when forming rir_i (0.020/0.026/0.026). The discrepancy is therefore unexplained, not merely unverified. Treat a rejection as meaningful and a non-rejection as weak evidence.

Fits sitting on the wrong-skew boundary are refused rather than answered: u^\hat u is then identical for every firm, so there is no estimated inefficiency for zz to explain, and the information matrix is singular there (Waldman 1982).

Value

A data frame with the corrected and naive statistics, their common degrees of freedom and p-values. The second-step coefficients and both sets of standard errors are attached as attributes.

References

Kim, M. and Schmidt, P. (2008). Valid tests of whether technical inefficiency depends on firm characteristics. Journal of Econometrics 144, 409–427.

Wooldridge, J. M. (2002). Econometric Analysis of Cross Section and Panel Data, pp. 139–141.

See Also

sfm (its uhet argument fits the alternative directly), marginal_effects, skewness_test

Examples

set.seed(2)
n <- 400
x1 <- rnorm(n); x2 <- rnorm(n)
z <- 0.9 * x1 + sqrt(1 - 0.81) * rnorm(n)   # correlated with x: G != 0
y <- 0.5 + 0.5 * x1 + 0.5 * x2 + rnorm(n, 0, 0.5) - abs(rnorm(n, 0, 1))
d <- data.frame(y, x1, x2, z)

f <- sfm(y ~ x1 + x2, model_name = "NHN", data = d, keep_objective = TRUE)
uhet_test(f, ~z, data = d)

U.S. fossil-fuel steam electric utilities, 1986-1997

Description

Panel data on U.S. investor-owned fossil fuel-fired steam electric utilities for the period 1986-1997. These data include measures of output, capital, labour and maintenance, and fuel.

Usage

data("USUtilities")

Format

A data frame with 972 observations (a balanced panel of observations on 81 utilities over 12 years) on the following 7 variables.

firmID

a numeric vector containing a unique firm identifier

year

a numeric vector containing the year of the observation

q

a numeric vector containing net steam electric power generation (MWh)

K

a numeric vector containing capital stock, calculated using a method described by Christensen and Jorgenson (1970)

L

a numeric vector containing quantity of labor and maintenance, calculated as cost divided by price index

F

a numeric vector containing quantity of fuel used, calculated as fuel costs divided by fuel price index

trend

a numeric vector containing an annual time trend (1992=100)

Details

The dataset covers 81 investor-owned utilities after aggregating subsidiaries and excluding plants in states with partial deregulation plans. Data sources include the Energy Information Administration (EIA), Federal Energy Regulatory Commission (FERC), and Bureau of Labor Statistics (BLS). Output is net steam electric generation from fossil fuel-fired boilers.

Source

Rungsuriyawiboon, S. and Stefanou, S.E. (2007). 'Dynamic Efficiency Estimation: An Application to U.S. Electric Utilities.' Journal of Business & Economic Statistics, 25(2), pp. 226-238. doi:10.1198/073500106000000288

References

Christensen, L.R. and Jorgenson, D.W. (1970). 'U.S. Real Product and Real Factor Input, 1928-1967.' Review of Income and Wealth, 16(1), pp. 19-50. doi: 10.1111/j.1475-4991.1970.tb00695.x

Examples

data(USUtilities)

Zero-Inefficiency Stochastic Frontier Models

Description

Fits the zero-inefficiency stochastic frontier model of Kumbhakar, Parmeter and Tsionas (2013): the sample is a mixture of fully efficient firms (u=0u = 0) and firms drawn from an ordinary normal-half-normal frontier.

  • "ZISF": the probability of being fully efficient is one constant for the whole sample.

  • "ZISF_Z": that probability depends on covariates, given in the second part of the formula (y ~ x | z).

Usage

zsfm(formula, model_name = c("ZISF", "ZISF_Z"), 
data, maxit.bobyqa = 10000,maxit.psoptim = 1000, maxit.optim = 1000, 
REPORT = 1, trace = 0, pgtol = 0,start_val = FALSE,PSopt = FALSE, 
optHessian = TRUE, inefdec = TRUE, upper = NA, 
Method = "L-BFGS-B",logit = TRUE,verbose=FALSE,
keep_objective = FALSE,rand.psoptim = NULL)

Arguments

formula

A symbolic description of the frontier, y ~ x1 + x2. For "ZISF_Z" a second part gives the covariates of the probability of full efficiency: y ~ x1 + x2 | z1.

model_name

"ZISF" or "ZISF_Z"; see ‘Description’. Matching ignores case, so "zisf" and "ZISF" are the same choice.

data

A data frame containing the variables named in formula.

maxit.bobyqa

Maximum number of iterations for the bobyqa optimization routine.

maxit.psoptim

Maximum number of iterations for the psoptim optimization routine.

maxit.optim

Maximum number of iterations for the optim optimization routine.

REPORT, trace, pgtol, upper

Accepted but not used by zsfm(): the optimizer's own tracing is controlled by verbose, and its bounds are set internally.

start_val

FALSE (the default) to build starting values from the data, or a numeric vector giving them directly, in the order of the rows of out.

PSopt

Logical. Run the particle-swarm stage (pso::psoptim) between bobyqa and optim? Defaults to FALSE.

optHessian

Logical. Finish with optim() and a numerically differentiated Hessian? Standard errors and t-values require TRUE, the default.

inefdec

Logical. TRUE (the default) for a production frontier, ε=v−u\varepsilon = v - u; FALSE for a cost frontier, ε=v+u\varepsilon = v + u.

Method

Method passed to optim for the final stage. Must be "L-BFGS-B": it is the only optim method that applies the lower and upper bounds this package uses to keep scale parameters positive, and any other value is refused with an error rather than quietly optimized without them.

logit

Logical, "ZISF_Z" only. The link from z′γz'\gamma to the probability of full efficiency; see ‘The probability of full efficiency’. Prefer the default, TRUE.

verbose

Logical. Print optimization progress messages? Default is FALSE.

keep_objective

Logical. Retain the likelihood closure on the fitted object as $objective, so the score-based tools can difference it: vcov(type = "bhhh") (OPG/BHHH standard errors), influence_sfa, the sandwich methods, TIC and vuong. New in 1.2.1. FALSE by default, because a closure carries its enclosing environment, so a fit saved with one serializes the estimation data alongside the results.

rand.psoptim

Integer. Seed for replication of psoptim. Defaults to NULL.

Details

The model

With probability pp a firm sits exactly on the frontier, so its composed error is noise alone, ε=v\varepsilon = v. With probability 1−p1 - p it is drawn from an ordinary normal-half-normal frontier. This is the natural specification when some fraction of the sample is thought to be fully efficient, which an ordinary frontier cannot represent: it forces u>0u > 0 almost surely.

The probability of full efficiency

Under "ZISF", pp is one constant, parameterized as p=exp⁡(−∣γ∣)p = \exp(-|\gamma|). That parameterization is exactly symmetric in γ\gamma: +γ+\gamma and −γ-\gamma give identical fits and the optimizer may return either, so the sign of the reported gamma carries no meaning.

Under "ZISF_Z", pp varies with the covariates zz through the link chosen by logit:

  • logit = TRUE (the default): the logistic link Λ(z′γ)\Lambda(z'\gamma), which covers (0,1)(0,1).

  • logit = FALSE: Φ(z′γ)/(1+Φ(z′γ))\Phi(z'\gamma)/(1 + \Phi(z'\gamma)), which is bounded above by 0.5. Under that link no observation can be assigned a probability of full efficiency above one half, whatever the data. It is not the probit analogue of the default and is retained only for backward compatibility.

Efficiency predictions

Two per-observation predictions are returned, and they answer different questions:

  • post.prob is the posterior probability that the observation is fully efficient, given its residual.

  • jlms is the Jondrow et al. (1982) prediction E[u∣ε]E[u \mid \varepsilon] conditional on the observation being in the inefficient regime. It is the ordinary normal-half-normal formula and does not weight by post.prob.

The prediction of uu over both regimes is (1 - post.prob) * jlms. The two can differ by orders of magnitude when the fitted pp is large, so use the product when you want E[u∣ε]E[u \mid \varepsilon] itself.

Iteration defaults

Like sfm(), zsfm() has a closed-form likelihood, so its optimization defaults (thousands of iterations) are much higher than psfm()'s, which uses simulated maximum likelihood for several models. See psfm's Details.

Value

An object of class "sfareg" containing the following components:

out

A matrix with parameter estimates, standard errors, and t-values. Rows are gamma, sigv, sigu and the frontier coefficients for "ZISF"; sigv, sigu, the frontier coefficients and then the γ\gamma coefficients for "ZISF_Z".

opt

A list containing the optimization results from the final optimization procedure.

total_time

The total computation time for model estimation.

start_v

The starting values used in the optimization.

model_name

"ZISF" or "ZISF_Z".

formula

The formula used in the model specification.

jlms

The Jondrow et al. (1982) inefficiency prediction E[u∣ε]E[u \mid \varepsilon] within the inefficient regime; see ‘Efficiency predictions’.

post.prob

Posterior probability that each observation is fully efficient.

coefficients

A vector of estimated parameters.

std.errors

A vector of standard errors for the estimated parameters (NA if optHessian = FALSE).

t.values

A vector of t-values for the estimated parameters (NA if optHessian = FALSE).

call

The matched call.

nobs

The number of observations used in the fit.

objective

The likelihood closure, when keep_objective = TRUE.

Note

Standard errors require optHessian = TRUE.

Author(s)

Christopher F. Parmeter and David H. Bernstein

References

Kumbhakar, S.C., Parmeter, C.F. and Tsionas, E.G. (2013) 'A zero inefficiency stochastic frontier model', Journal of Econometrics, 172(1), pp. 66-76. doi:10.1016/j.jeconom.2012.08.021.

Jondrow, J., Lovell, C.A.K., Materov, I.S. and Schmidt, P. (1982) 'On the estimation of technical inefficiency in the stochastic frontier production function model', Journal of Econometrics, 19(2-3), pp. 233-238.

See Also

sfm, psfm, ttsfm, sfareg-methods, panel89

Examples

library(sfa)  

eqz     <- y ~ q1 + q2 + q3 + q4 + q5 + w1 + w2 + w3 + w4 | z

data(panel89)

f <- zsfm(formula    = eqz,
          model_name = "ZISF_Z",
          data       = panel89,
          logit      = TRUE)
f

## Inefficiency over both regimes, not just the inefficient one
head((1 - f$post.prob) * f$jlms)