--- title: "Selective bound testing at interim analyses" author: "Keaven Anderson" output: rmarkdown::html_vignette bibliography: gsDesign.bib vignette: > %\VignetteIndexEntry{Selective bound testing at interim analyses} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- ```{r, include=FALSE} knitr::opts_chunk$set( collapse = TRUE, comment = "#>", dev = "svg", fig.ext = "svg", fig.width = 7.2916667, fig.asp = 0.618, fig.align = "center", out.width = "80%" ) options(width = 68) ``` ## Introduction ```{r, echo=FALSE, message=FALSE, warning=FALSE} library(gsDesign) library(knitr) ``` All examples use a common time-to-event design. Enrollment lasts 16 months, with the relative enrollment rate increasing over the first 6 months and then remaining constant. The trial lasts 36 months. Control-group survival is 50%, 40%, and 35% at 1, 2, and 3 years, respectively. We convert each interval's change in log survival to a piecewise exponential failure rate. Dropout is exponential at an annual rate of 0.02, and the hazard ratio under the alternative is 0.7. Analyses occur after 40%, 70%, and 100% of planned events. ```{r} survival_time <- c(0, 12, 24, 36) control_survival <- c(1, .50, .40, .35) control_failure_rate <- -diff(log(control_survival)) / diff(survival_time) enrollment_duration <- c(rep(1, 5), 11) relative_enrollment_rate <- 1:6 survival_args <- list( k = 3, alpha = .025, beta = .1, hr = .7, timing = c(.4, .7), sfu = sfLDOF, sfl = sfHSD, sflpar = 1, lambdaC = control_failure_rate, S = c(12, 24), eta = .02 / 12, gamma = relative_enrollment_rate, R = enrollment_duration, T = 36, minfup = 20 ) ``` Equivalently, `gsDesign2::s2pwe(times = c(12, 24, 36), survival = c(.50, .40, .35))$rate` performs this survival-to-rate conversion. Every example below inherits `timing = c(.4, .7)` and these survival and enrollment assumptions from `survival_args`. The `gamma` input changes from 1 to 6 over the first 6 months; its sixth value then applies through month 16. Because total duration and minimum follow-up are fixed, `gsSurv()` scales this relative enrollment-rate pattern to attain 90% power. Designs with different stopping rules can therefore show different fitted enrollment rates while retaining the same timing, survival, dropout, and enrollment-ramp assumptions. The Lan--DeMets O'Brien--Fleming spending function, `sfLDOF`, sets a high bar for early efficacy stopping that is generally acceptable to regulators. The first interim is delayed until 40% of planned events to balance useful early futility stopping against power loss. We initially use `sfHSD` with `sflpar = 1`; this is a relatively aggressive starting point that should be examined in sensitivity analyses before adopting the design. In many clinical trial designs, it is desirable to test only certain boundaries at specific interim analyses. For example: - **Futility testing at early interims only**: In some trial designs, a futility assessment is only meaningful at the first interim analysis. Later analyses focus solely on efficacy. - **No efficacy testing at the first interim**: Some regulatory or operational considerations may preclude testing for efficacy at the very first interim. - **Selective harm monitoring**: For designs with harm bounds (`test.type = 7` or `8`), harm monitoring may be needed only at certain analyses. The `testUpper`, `testLower`, and `testHarm` parameters in `gsDesign()`, `gsSurv()`, and `gsSurvCalendar()` allow fine-grained control over which bounds are active at each analysis. When a bound is inactive at a given analysis, it is set to an extreme value ($\pm 20$ on the Z-scale) so that it cannot be crossed. Every characteristic for that bound, including cumulative crossing probability, is displayed as `NA` at the inactive analysis. The underlying probability arrays retain the cumulative crossing information. ## Parameters Each of `testUpper`, `testLower`, and `testHarm` accepts either a single logical value (recycled to all analyses) or a logical vector of length `k` (the number of analyses): | Parameter | Description | Default | Constraints | |-----------|-------------|---------|-------------| | `testUpper` | Test the upper (efficacy) bound | `TRUE` | Must be `TRUE` at the final analysis. For `test.type` 1 and 2, overridden to all `TRUE`. | | `testLower` | Test the lower (futility) bound | `TRUE` | Ignored for `test.type = 1` (one-sided). Overridden to all `TRUE` for `test.type = 2` (symmetric). For `test.type >= 3`, at least one analysis must have `testLower = TRUE`. | | `testHarm` | Test the harm bound | `TRUE` | Only applies to `test.type = 7` or `8`. At least one analysis must have `testHarm = TRUE`. | **Validation**: At every analysis, at least one of the active bounds must be `TRUE`. If all three are `FALSE` at any analysis, an error is raised. ## Example 1: Futility testing only at the first interim A common scenario is to test for futility only at the first interim analysis. This is useful when the trial's data monitoring committee wants an early "go/no-go" decision, but not ongoing futility monitoring. For this non-binding `test.type = 4` design, efficacy testing also begins at IA2 rather than IA1. ```{r} # Non-binding futility only at IA1; efficacy starts at IA2 x1 <- do.call(gsSurv, c(survival_args, list( test.type = 4, testUpper = c(FALSE, TRUE, TRUE), testLower = c(TRUE, FALSE, FALSE) ))) ``` The active futility bound depends on the beta spending function as well as the testing schedule. Here, `sfl = sfHSD` and `sflpar = 1` specify the spending pattern. Changing `sfl` or `sflpar` changes how beta is allocated and therefore changes the futility bound at IA1. With `sflpar = 1`, the futility rule is relatively aggressive: the IA1 bound is approximately HR 0.912, and the probability of stopping there under the design alternative, HR 0.7, is 5.2%. Its impact on power, planned events, and expected trial duration should be compared with less aggressive choices. The lower bound is active only at IA1. At IA2 and the final analysis, the futility bound shows as `NA`: ```{r} gsBoundSummary(x1, exclude = NULL) |> gt::gt() ``` The probabilities under the null and alternative are recomputed accounting for the inactive bounds. The underlying cumulative futility crossing probability does not increase after IA1 since no further futility testing occurs; the later inactive futility rows in `gsBoundSummary()` are displayed entirely as `NA`. Using `exclude = NULL` includes conditional power at the current trend (`CP`), conditional power under the design alternative (`CP H1`), and predictive power (`PP`). Predictive power averages conditional power over the posterior formed from the interim result and the `prior` argument; it is sometimes referred to as average conditional power. By default, `gsBoundSummary()` uses a normal prior centered halfway between the null and alternative, with standard deviation `10 / sqrt(x$n.fix)`. Its variance is equivalent to observing 1% of the fixed-design information, so it is intended to be relatively weak. This means that the posterior used to determine predictive power is dominated by the interim data and primarily reflects variability in the interim observed treatment effect. ## Example 2: Asymmetric lower testing For `test.type = 6`, `astar` controls Type I error allocated to the lower bound. To be somewhat aggressive, we set `astar = 0.15`. As with the other non-binding designs considered here, we skip efficacy testing at IA1: ```{r} xs <- do.call(gsSurv, c(survival_args, list( test.type = 6, astar = .15, testUpper = c(FALSE, TRUE, TRUE), testLower = c(TRUE, TRUE, FALSE) ))) gsBoundSummary(xs, exclude = NULL) |> gt::gt() ``` ## Example 3: Selective harm monitoring (test.type 7/8) For designs with a separate harm bound, the `testHarm` parameter controls which analyses include harm monitoring. This is the same concept as `testLower` for `test.type = 5` or `6`, in that the harm bound tests whether experimental treatment is worse than control. For `test.type = 7` or `8`, the futility bound instead examines differences from the hypothesized effect `hr`. This can be useful when harm monitoring is most critical during early enrollment, before longer-term safety data are available. Continuing with the common survival assumptions, we change the design type and add a harm spending function and testing schedule. We use the same Hwang--Shih--DeCani parameter, 1, for harm spending and skip efficacy testing at IA1: ```{r} # Harm bound design with harm monitoring only at IA1 and IA2 xh <- do.call(gsSurv, c(survival_args, list( test.type = 8, astar = .15, sfharm = sfHSD, sfharmparam = 1, testUpper = c(FALSE, TRUE, TRUE), testHarm = c(TRUE, TRUE, FALSE) ))) gsBoundSummary(xh, exclude = NULL) |> gt::gt() ``` The harm bound is `NA` at the final analysis. ### Why skipping final futility can change an earlier harm bound For `test.type = 7` and `8`, harm and futility are mutually exclusive lower-tail stopping outcomes. When both bounds are active, observations at or below the harm bound are classified as harm stops, while observations between the harm and futility bounds are classified as futility-only stops. If futility is inactive but harm remains active, harm becomes the only active lower stopping boundary at that analysis. This distinction applies even to `test.type = 8`. Non-binding means that futility and harm stopping are ignored when Type I error is protected; it does not mean that the reported harm and futility boundaries are derived independently. The boundaries are jointly calibrated to their spending targets and active testing schedules. The following three designs skip efficacy testing at IA1. They differ only in whether futility and harm are tested at the final analysis: ```{r} common_harm_args <- c(survival_args, list( test.type = 8, astar = .15, testUpper = c(FALSE, TRUE, TRUE), sfharm = sfHSD, sfharmparam = 1 )) x_all_lower <- do.call(gsSurv, c( common_harm_args, list( testLower = c(TRUE, TRUE, TRUE), testHarm = c(TRUE, TRUE, TRUE) ) )) x_final_harm <- do.call(gsSurv, c( common_harm_args, list( testLower = c(TRUE, TRUE, FALSE), testHarm = c(TRUE, TRUE, TRUE) ) )) x_matched_lower <- do.call(gsSurv, c( common_harm_args, list( testLower = c(TRUE, TRUE, FALSE), testHarm = c(TRUE, TRUE, FALSE) ) )) harm_comparison <- data.frame( Analysis = seq_len(x_all_lower$k), `All lower bounds active` = x_all_lower$harm$bound, `Final harm only` = x_final_harm$harm$bound, `Both final lower bounds skipped` = x_matched_lower$harm$bound, check.names = FALSE ) kable(harm_comparison, digits = 3, caption = "Harm bounds by final-analysis testing schedule") ``` When final futility is skipped but final harm remains active, the lower-tail stopping partition changes. The joint calibration can therefore change an earlier harm bound even though `testHarm` itself was not changed. This is expected when harm monitoring is genuinely intended at the final analysis. If neither futility nor harm will be assessed at the final analysis, specify both schedules explicitly: ```r testLower = c(TRUE, TRUE, FALSE) testHarm = c(TRUE, TRUE, FALSE) ``` Matching these schedules leaves the earlier harm bounds unchanged relative to the design with all lower bounds active. Conversely, retain `testHarm = c(TRUE, TRUE, TRUE)` when a final harm assessment is intended and interpret the recalibrated harm bounds as part of that testing strategy. ## Example 4: Combining selective efficacy and futility Both `testUpper` and `testLower` can be specified simultaneously. For example, a design with futility-only at IA1 and efficacy-only at IA2: ```{r} # Futility only at IA1; efficacy at IA2 and Final x5 <- do.call(gsSurv, c(survival_args, list( test.type = 4, testUpper = c(FALSE, TRUE, TRUE), testLower = c(TRUE, FALSE, FALSE) ))) gsBoundSummary(x5, exclude = NULL) |> gt::gt() ``` Note the `NA` values: efficacy is `NA` at IA1, and futility is `NA` at IA2 and the final analysis. ## Validation rules The following rules are enforced: 1. **`testUpper` must be `TRUE` at the final analysis** (the trial must always be able to reject $H_0$). 2. **At least one bound must be active at every analysis**. For example, setting `testUpper = c(FALSE, TRUE, TRUE)` and `testLower = c(FALSE, TRUE, TRUE)` would fail because no bound is active at IA1. 3. **`test.type = 1`**: Only one-sided efficacy testing. `testLower` is ignored (set to `FALSE` internally). 4. **`test.type = 2`**: Symmetric two-sided testing. Both `testUpper` and `testLower` are overridden to all `TRUE`. 5. **`test.type` 3--8**: `testLower` must be `TRUE` for at least one analysis. 6. **`test.type` 7 and 8**: `testHarm` must be `TRUE` for at least one analysis. ```{r, error=TRUE} # This fails: testUpper must be TRUE at the final analysis try(do.call(gsSurv, c(survival_args, list( test.type = 3, testUpper = c(TRUE, TRUE, FALSE) )))) ``` ```{r, error=TRUE} # This fails: no bound active at analysis 1 try(do.call(gsSurv, c(survival_args, list( test.type = 4, testUpper = c(FALSE, TRUE, TRUE), testLower = c(FALSE, TRUE, TRUE) )))) ``` ## Accessing stored flags The `testUpper`, `testLower`, and `testHarm` logical vectors are stored on the returned `gsDesign` object: ```{r} x1$testUpper x1$testLower x1$testHarm ``` These can be inspected programmatically for downstream analyses or reporting. For `gsSurv()` and `gsSurvCalendar()` objects, `x$call` records the original call and `x$inputs` retains evaluated survival-model inputs plus the applicable testing-schedule arguments. The normalized logical vectors above remain the authoritative schedule used in calculations and summaries. ## Type I Error Preservation A key property of the selective bounds implementation is that **Type I error is preserved at the nominal level** regardless of which analyses are skipped. ### How it works When bounds are selectively deactivated, the cumulative spending at each *performed* analysis remains at the spending function's planned value. At inactive analyses, the cumulative spending is frozen (no incremental spend), causing the C code to produce $\pm$EXTREMEZ bounds. At the next active analysis, the incremental spend absorbs the budget from any prior skipped analyses, so the cumulative spend catches up to the planned level. The efficacy bounds at active analyses are then recomputed using the modified spending, ensuring the total alpha spent equals the nominal level. If information was not fixed on input, the maximum information is subsequently solved again using the boundaries that will actually be tested so that power remains at its target. ### Non-binding futility (test.type 4 or 6) For non-binding designs, the efficacy bounds are computed under the assumption that the trial *may continue past the futility bound* (i.e., futility does not contribute to the upper alpha calculation). Since upper bounds are independent of lower bounds, removing futility has no effect on the upper (efficacy) bounds or the non-binding alpha: ```{r} # Baseline non-binding design; efficacy starts at IA2 x_nb <- do.call(gsSurv, c(survival_args, list( test.type = 4, testUpper = c(FALSE, TRUE, TRUE) ))) # Remove futility at IA2 and final x_nb_sel <- do.call(gsSurv, c(survival_args, list( test.type = 4, testUpper = c(FALSE, TRUE, TRUE), testLower = c(TRUE, FALSE, FALSE)) )) # Non-binding alpha (computed ignoring lower bounds) nb_alpha_base <- sum(gsDesign:::gsprob(0, x_nb$n.I, rep(-20, 3), x_nb$upper$bound, r = x_nb$r)$probhi) nb_alpha_sel <- sum(gsDesign:::gsprob(0, x_nb_sel$n.I, rep(-20, 3), x_nb_sel$upper$bound, r = x_nb_sel$r)$probhi) cat("Baseline non-binding alpha: ", nb_alpha_base, "\n") cat("Selective non-binding alpha:", nb_alpha_sel, "\n") cat("Upper bounds identical: ", all.equal(x_nb$upper$bound, x_nb_sel$upper$bound), "\n") ``` For the planned design, omitting efficacy at IA1 redistributes its alpha to the active analyses while still totalling exactly $\alpha = 0.025$: ```{r} nb_alpha_eff <- sum(gsDesign:::gsprob( 0, x_nb$n.I, rep(-20, 3), x_nb$upper$bound, r = x_nb$r )$probhi) cat("Non-binding alpha (skip IA1 efficacy):", nb_alpha_eff, "\n") ``` ### Binding futility (test.type 3 or 5) For binding designs, the efficacy bounds depend on the futility bounds. When futility bounds are selectively removed, the bounds are recomputed with the modified spending. For a newly derived design, information is then adjusted to retain target power; when `n.I` is supplied, information remains fixed. This preserves cumulative Type I error at the nominal level while respecting the requested power calculation: ```{r} # Baseline binding design x_b <- do.call(gsSurv, c(survival_args, list(test.type = 3))) cat("Baseline alpha:", sum(x_b$upper$prob[, 1]), "\n") # Remove futility at IA2 and final x_b_sel <- do.call(gsSurv, c(survival_args, list( test.type = 3, testLower = c(TRUE, FALSE, FALSE)) )) cat("Selective alpha:", sum(x_b_sel$upper$prob[, 1]), "\n") cat("Selective power:", sum(x_b_sel$upper$prob[, 2]), "\n") # Remove efficacy at IA1 x_b_eff <- do.call(gsSurv, c(survival_args, list( test.type = 3, testUpper = c(FALSE, TRUE, TRUE)) )) cat("Skip IA1 efficacy alpha:", sum(x_b_eff$upper$prob[, 1]), "\n") cat("Skip IA1 efficacy power:", sum(x_b_eff$upper$prob[, 2]), "\n") ``` In all cases, the actual Type I error is exactly $\alpha = 0.025$ (within numerical tolerance). The bounds at active analyses adjust to allocate the spending budget, and derived information is recalibrated so that skipped efficacy or lower analyses do not leave the design over-powered. When `gsBoundSummary()` is used with alternate alpha levels, every efficacy column retains the original `testUpper` schedule. An efficacy look that was not planned is not reintroduced merely because another alpha level is displayed. Alternate-alpha summaries are available for the one-sided or non-binding test types 1, 4, 6, and 8. They are intentionally not offered for binding types 2, 3, 5, or 7, which are outside the Maurer--Bretz non-binding multiple-testing framework.