Generalized Maximum Entropy Estimation for Kink Regression with Multiple Covariates

Description

Estimates a kink (threshold) regression model using the Generalized Maximum Entropy (GME) framework. The model allows for a nonlinear relationship in a primary regressor and linear effects from additional covariates.

The kink regression model is specified as:

y_i = \beta_0 + \beta_1 (x_i - \tau)^{-} + \beta_2 (x_i - \tau)^{+} + \mathbf{Z}_i' \gamma + \varepsilon_i,

where (x_i - \tau)^{-} = (x_i - \tau)\mathbf{1}(x_i < \tau) and (x_i - \tau)^{+} = (x_i - \tau)\mathbf{1}(x_i \geq \tau) represent the negative and positive parts of the kink function, respectively. The parameter \tau denotes the threshold (kink point).

Usage

MEkink(y, x, Z = NULL,
       number = "3",
       entropy = "shannon",
       s1 = 10,
       s2 = 3,
       boot = FALSE, B = 100)

Arguments

Details

The Generalized Maximum Entropy (GME) approach estimates the unknown parameters by representing them as probability distributions over discrete support spaces. These probabilities are obtained by maximizing an entropy measure (e.g., Shannon entropy) subject to the consistency constraints imposed by the regression model.

The argument number controls the number of support points used for discretization, while s1 and s2 define the bounds for the coefficient and error supports, respectively.

When boot = TRUE, standard errors are computed using bootstrap resampling, which is particularly useful in nonlinear and nonstandard estimation settings such as kink regression.

Value

A list with the following components:

Author(s)

Woraphon Yamaka, Paravee Maneejuk, and Nuttaphong Kaewtathip

References

Golan, A., Judge, G. G., & Miller, D. (1996). Maximum entropy econometrics. Iowa State University Press.

Jaynes, E. T. (1957). Information theory and statistical mechanics. Physical Review, 106(4), 620–630.

Shannon, C. E. (1948). A mathematical theory of communication. Bell System Technical Journal, 27, 379–423.

Examples

## Simulated data
set.seed(1)
n <- 15
thres <- 3
x <-rnorm(n, thres, 5)
z1 <- rnorm(n)
z2 <- rnorm(n)

Z <- cbind(z1, z2)

y <- 0.5 +
     1*(x*(x < 3)) +
    -1*(x*(x >= 3)) +
     rnorm(n, sd=0.2)

## Estimate model
fit <- MEkink(y, x, Z,number="5", entropy="shannon",s1=10,s2=3,boot=FALSE)
fit

Generalized Maximum Entropy for estimating linear regression

Description

Estimates a linear regression model using the Generalized Maximum Entropy (GME) framework.

The model is given by:

y_i = \beta_0 + \sum_{j=1}^{k} \beta_j x_{ij} + \varepsilon_i

The parameters are estimated by representing unknown coefficients and disturbances as probability distributions over discrete support spaces.

Usage

MEregress(y,x,number,entropy,s1,s2,boot,B)

Arguments

Details

The GME approach represents unknown parameters and disturbances as probability distributions over discrete support spaces, and estimates them by maximizing Shannon entropy subject to the model constraints.

The number of support points is controlled by the number argument, which determines the discretization of the support space (e.g., "3", "5", or "7" points). A larger number of support points provides greater flexibility in approximating the underlying distributions, but increases computational complexity.

The bounds of the support spaces are specified by s1 for the coefficients and s2 for the error term. These bounds incorporate prior information and play an important role in shaping the estimated distributions.

Compared to traditional methods such as ordinary least squares, the GME estimator is particularly useful in ill-posed problems and settings with limited data, where distributional assumptions may be unreliable.

Value

Author(s)

Woraphon Yamaka, Paravee Maneejuk, and Nuttaphong Kaewtathip

References

Golan, A., Judge, G. G., and Miller, D. (1996). Maximum Entropy Econometrics. Wiley.

Jaynes, E. T. (1957). Information theory and statistical mechanics. Physical Review, 106(4), 620–630.

Maneejuk, P., Yamaka, W., and Sriboonchitta, S. (2019). Does the Kuznets curve exist in Thailand? Annals of Operations Research.

Maneejuk, P. and Yamaka, W. (2020). Entropy inference in smooth transition kink regression.

Examples

set.seed(1)
n=20
e=rnorm(n)
x0=rnorm(n)
x1=rnorm(n)
y=1+2*x0+3*x1+e
x=cbind(x0,x1)
MEregress(y,x,number="3",entropy = "shannon",s1=10,s2=5,boot=FALSE)

Generalized Maximum Entropy Estimation for Smooth Transition Kink Regression

Description

Estimates a smooth transition kink regression model using the Generalized Maximum Entropy (GME) framework.

The model is defined as:

y_i = \beta_0 + \beta_1 x_i (1 - G(x_i; \gamma, \tau)) + \beta_2 x_i G(x_i; \gamma, \tau) + \varepsilon_i,

where G(x_i; \gamma, \tau) = \frac{1}{1 + \exp(-\gamma (x_i - \tau))} is a logistic transition function.

Unknown parameters and disturbances are represented as probability distributions over discrete support spaces, providing robust estimation under ill-posed or ill-conditioned settings.

Usage

MEskink(y,x,Z,number,entropy,s1,s2,boot,B)

Arguments

Details

The Generalized Maximum Entropy (GME) approach provides a flexible framework for estimating models in ill-posed or underdetermined settings. The method is based on the principle of maximum entropy introduced by Jaynes (1957), which selects probability distributions that are maximally non-committal with respect to missing information.

In this framework, unknown parameters and disturbances are represented as discrete probability distributions over predefined support spaces. These probabilities are estimated by maximizing Shannon entropy (Shannon, 1948) subject to the consistency constraints implied by the regression model.

The approach was further developed by Golan, Judge, and Miller (1996) for econometric applications, allowing for estimation without imposing strict distributional assumptions on the error term. As a result, the GME estimator is robust to multicollinearity, small sample sizes, and ill-conditioned problems.

This implementation focuses on a smooth transition kink regression model with a single explanatory variable, where nonlinear behavior is captured through a logistic transition function.

Value

Author(s)

Woraphon Yamaka, Paravee Maneejuk, and Nuttaphong Kaewtathip

References

Golan, A., Judge, G. G., and Miller, D. (1996). Maximum Entropy Econometrics. Wiley.

Jaynes, E. T. (1957). Information theory and statistical mechanics. Physical Review, 106(4), 620–630.

Maneejuk, P., Yamaka, W., and Sriboonchitta, S. (2019). Does the Kuznets curve exist in Thailand? Annals of Operations Research.

Maneejuk, P. and Yamaka, W. (2020). Entropy inference in smooth transition kink regression.

Examples

set.seed(1)
n <- 20
thres <- 3
gam <- 1.2
e <- rnorm(n)
x <-rnorm(n, thres, 5)
z1 <- rnorm(n)
z2 <- rnorm(n)

Z <- cbind(z1, z2)
alpha <- c(0.5, 1, -1)

g <- 1/(1+exp(-gam*(x-thres)))

y <- alpha[1] +
     alpha[2]*(x*(1-g)) +
     alpha[3]*(x*g) + e

fit2=MEskink(y, x, Z,number = "7",entropy="tsallis", s1 = 10, s2 = 5,boot=FALSE)
fit2

Plot Method for Smooth Transition and Kink Regression Models

Description

Plots the observed data together with the fitted regression curve from a Generalized Maximum Entropy kink or smooth transition kink regression model.

Usage

## S3 method for class 'MEskink'
plot(x, type = c("smooth", "kink"), main = NULL, ...)

Arguments

Details

This function provides a graphical representation of the relationship between the dependent variable and the explanatory variable.

For type = "smooth", the fitted curve is based on a smooth transition kink regression model using a logistic transition function, allowing the slope to change gradually around the estimated threshold.

For type = "kink", the fitted curve corresponds to a standard kink (piecewise linear) regression, where the slope changes abruptly at the estimated threshold.

The function constructs a grid over the range of the explanatory variable and evaluates the fitted model to generate the regression curve.

For smooth models, the input object must contain a smooth parameter.

Value

This function returns NULL. It is used for its side effect of producing a plot.

Author(s)

Woraphon Yamaka, Paravee Maneejuk, and Nuttaphong Kaewtathip

Examples

set.seed(1)
n <- 20
thres <- 3
gam <- 1.2
e <- rnorm(n)
x <- rnorm(n, thres, 5)
z1 <- rnorm(n)
z2 <- rnorm(n)

Z <- cbind(z1, z2)
alpha <- c(0.5, 1, -1)

g <- 1 / (1 + exp(-gam * (x - thres)))

y <- alpha[1] +
     alpha[2] * (x * (1 - g)) +
     alpha[3] * (x * g) + e

fit2 <- MEskink(y, x, Z,
                number = "7",
                entropy = "tsallis",
                s1 = 10, s2 = 5,
                boot = FALSE)

plot(fit2, type = "smooth")