Introduction to LiblineaR

Choosing a type

type selects both the loss function and the regularization. Two families:

Classification (type 0-7):

Regression (type 11-13), all L2-regularized support vector regression:

Rules of thumb:

• For an ordinary classification problem, type=0 (logistic regression) or type=2 (L2-loss SVM) are the usual defaults; both give one weight vector per class for multi-class problems via one-vs-rest, except type=4, which always fits one weight vector per class simultaneously (relevant if you read $W’s shape — see below).
• type=1/2 and type=0/7 are the dual/primal formulations of the same problem respectively, and converge to (numerically close to) the same model — a useful sanity check if you’re unsure which to trust.
• L1-regularized types (5, 6) push weights toward exact zero — useful for feature selection on high-dimensional data.
• For regression, type=11 (primal) is usually fastest; type=12/13 differ in whether large errors are penalized quadratically or linearly.

x <- iris[, 1:4]
y <- iris[, 5]

m_lr  <- LiblineaR(x, y, type = 0)   # logistic regression
m_svm <- LiblineaR(x, y, type = 2)   # L2-loss SVM

dim(m_lr$W)   # one row per class (3 classes, multi-class problem)
#> [1] 3 5

bias, epsilon, svr_eps: what the defaults actually mean

bias (default 1): if bias > 0, every row gets an extra constant feature appended with that value ([data; bias]) — this is what lets the model fit an intercept. If bias <= 0, no bias term is added at all (the decision boundary is forced through the origin). For backward compatibility, bias=TRUE/FALSE are also accepted (TRUE behaves like 1, FALSE like 0, i.e. no bias).

epsilon (default NULL): the solver’s stopping tolerance. Leave it at the default — NULL lets LIBLINEAR apply its own per-solver default (0.01 for primal solvers, 0.1 for dual solvers; these differ because primal and dual solvers measure convergence on different quantities). Passing a numeric value overrides that for every solver uniformly, which is rarely what you want unless you’re deliberately trading convergence tightness for speed.

svr_eps (regression only, default 0.1 if left NULL): the width of the epsilon-insensitive tube — errors smaller than this aren’t penalized at all. There’s no universally good default; it depends on the scale of your target variable, so it’s worth setting explicitly for regression:

xr <- as.matrix(iris[, 1:3])
yr <- iris[, 4]
m_svr <- LiblineaR(xr, yr, type = 11, svr_eps = 0.05)

Sparse input

data (and predict()’s newx) accept dense matrices/data frames, or sparse matrices of class matrix.csr/matrix.csc/matrix.coo (package SparseM) or dgCMatrix/dgRMatrix/dgTMatrix (package Matrix). The type is detected automatically — no separate argument needed. All six sparse classes and dense input give identical coefficients and predictions on the same data; pick whichever integrates better with the rest of your pipeline.

if (requireNamespace("Matrix", quietly = TRUE)) {
  x_sparse <- Matrix::Matrix(as.matrix(x), sparse = TRUE)
  m_sparse <- LiblineaR(x_sparse, y, type = 0)
  identical(dim(m_sparse$W), dim(m_lr$W))
}
#> 'as(<matrix>, "dgRMatrix")' is deprecated.
#> Use 'as(as(as(., "dMatrix"), "generalMatrix"), "RsparseMatrix")' instead.
#> See help("Deprecated") and help("Matrix-deprecated").
#> [1] TRUE

Class weighting with wi

wi reweights each class’s effective regularization constant (C_class = cost * wi[class], default weight 1 for every class not named). This is the tool for imbalanced data: naming only the minority class with a higher weight pushes the solver to trade some overall accuracy for better recall on that class — a deliberate trade-off, not a bug, and one you should expect to see reflected in a lower raw accuracy alongside a better balanced accuracy.

# Not all classes need to be named -- only the one(s) you want to reweight.
m_weighted <- LiblineaR(x, y, type = 0, wi = c(setosa = 5))

Finding a good cost: heuristicC(), cross, and findC

Three complementary tools:

• heuristicC(data): a fast, closed-form heuristic (Joachims’ SVM-light heuristic) giving a reasonable starting point for cost, computed directly from the data with no training involved.
• cross=k: runs k-fold cross-validation at the given cost and returns the CV accuracy (classification) or MSE (regression) as a single number — no model object. Useful for evaluating one specific cost.
• findC=TRUE: automatic search for a good cost, using repeated cross-validation internally. Only supported for type=0 and type=2 (the primal L2-regularized solvers); any other type raises an error. Returns the best cost found, not a model — retrain with that value to get the actual model.

co <- heuristicC(x)
co
#> [1] 0.1274724

acc <- LiblineaR(x, y, type = 0, cost = co, cross = 5)
acc
#> [1] 0.8533333

best_cost <- LiblineaR(x, y, type = 0, findC = TRUE, cross = 5)
best_cost
#> [1] 256

m_final <- LiblineaR(x, y, type = 0, cost = best_cost)

Predicting

predict() accepts a vector (single feature) or a matrix/data frame with the same columns as training — reordered and with any extra columns dropped automatically, matched by column name if newx has names.

p <- predict(m_final, x)
mean(as.character(p$predictions) == as.character(y))
#> [1] 0.98

# Probabilities are only available for logistic regression (type 0, 6, 7).
p_proba <- predict(m_final, x, proba = TRUE)
head(p_proba$probabilities)
#>         setosa versicolor    virginica
#> [1,] 0.9193942 0.08060579 9.586857e-16
#> [2,] 0.7786324 0.22136764 1.461904e-14
#> [3,] 0.8535065 0.14649348 6.496319e-15
#> [4,] 0.7914231 0.20857690 4.005768e-14
#> [5,] 0.9355732 0.06442682 8.260311e-16
#> [6,] 0.9756365 0.02436346 2.803273e-15