{ "cells": [ { "cell_type": "markdown", "id": "4ec506f4", "metadata": {}, "source": [ "# Multiple Comparisons Correction" ] }, { "cell_type": "code", "execution_count": 2, "id": "e3de31be", "metadata": { "execution": { "iopub.execute_input": "2026-09-30T09:43:33.436931Z", "iopub.status.busy": "2026-09-30T09:43:33.436657Z", "iopub.status.idle": "2026-09-30T09:43:35.733331Z", "shell.execute_reply": "2026-09-30T09:43:35.733018Z" }, "tags": [ "hide-cell" ] }, "outputs": [], "source": "# First, import the necessary modules and functions\nimport os\n\nimport matplotlib.pyplot as plt\nfrom myst_nb import glue\nfrom nilearn import plotting\nfrom repo2data.repo2data import Repo2Data\n\n# Install the data if running locally, or points to cached data if running on neurolibre\nDATA_REQ_FILE = os.path.abspath(\"../binder/data_requirement.json\")\nrepo2data = Repo2Data(DATA_REQ_FILE)\ndata_path = repo2data.install()\ndata_path = os.path.join(data_path[0], \"data\")\n\n# Set an output directory for any files generated during the book building process\nout_dir = os.path.abspath(\"../outputs/\")\nos.makedirs(out_dir, exist_ok=True)" }, { "cell_type": "markdown", "id": "11b874fe", "metadata": {}, "source": [ "In NiMARE, multiple comparisons correction is separated from each CBMA and IBMA `Estimator`, so that any number of relevant correction methods can be applied after the `Estimator` has been fit to the `Dataset`.\n", "Some correction options, such as the `montecarlo` option for FWE correction, are designed to work specifically with a given `Estimator` (and are indeed implemented within the `Estimator` class, and only called by the `Corrector`).\n", "\n", "`Corrector`s are divided into two subclasses: `FWECorrector`s, which correct based on family-wise error rate, and `FDRCorrector`s, which correct based on false discovery rate.\n", "\n", "All `Corrector`s are initialized with a number of parameters, including the correction method that will be used.\n", "After that, you can use the `transform` method on a `MetaResult` object produced by a CBMA or IBMA `Estimator` to apply the correction method.\n", "This will return an updated `MetaResult` object, with both the statistical maps from the original `MetaResult`, as well as new, corrected maps.\n", "\n", "Here we will apply both FWE and FDR correction to results from a MKDADensity meta-analysis, performed back in [](content:cbma:mkdad).\n", "\n", "```{warning}\n", "In the following example, we use 5000 iterations for Monte Carlo FWE correction.\n", "Normally, one would use at least 10000 iterations, but we reduced this for the sake of speed.\n", "```" ] }, { "cell_type": "code", "execution_count": 3, "id": "faa18d17", "metadata": { "execution": { "iopub.execute_input": "2026-09-30T09:43:35.738439Z", "iopub.status.busy": "2026-09-30T09:43:35.737719Z", "iopub.status.idle": "2026-09-30T11:33:06.829182Z", "shell.execute_reply": "2026-09-30T11:33:06.828705Z" } }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "INFO:nimare.correct:Using correction method implemented in Estimator: nimare.meta.cbma.mkda.MKDADensity.correct_fwe_montecarlo.\n" ] }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "a501a63690054e9185fc5cfa33d6d211", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/5000 [00:00" }, "metadata": { "scrapbook": { "mime_prefix": "application/papermill.record/", "name": "figure_corr_cbma" } }, "output_type": "display_data" } ], "source": "#| label: figure_corr_cbmacell\nmeta_results = {\n \"Cluster-level Monte Carlo\": mc_results.get_map(\n \"z_desc-size_level-cluster_corr-FWE_method-montecarlo\",\n return_type=\"image\",\n ),\n \"Independent FDR\": fdr_results.get_map(\n \"z_corr-FDR_method-indep\",\n return_type=\"image\",\n ),\n}\n\nfig, axes = plt.subplots(figsize=(6, 4), nrows=2)\n\nfor i_meta, (name, file_) in enumerate(meta_results.items()):\n display = plotting.plot_stat_map(\n file_,\n annotate=False,\n axes=axes[i_meta],\n draw_cross=False,\n cmap=\"Reds\",\n cut_coords=[0, 0, 0],\n figure=fig,\n )\n axes[i_meta].set_title(name)\n\n colorbar = display._cbar\n colorbar_ticks = colorbar.get_ticks()\n if colorbar_ticks[0] < 0:\n new_ticks = [colorbar_ticks[0], 0, colorbar_ticks[-1]]\n else:\n new_ticks = [colorbar_ticks[0], colorbar_ticks[-1]]\n colorbar.set_ticks(new_ticks, update_ticks=True)\n\nglue(\"figure_corr_cbma\", fig, display=False)" }, { "cell_type": "markdown", "id": "78136106", "metadata": {}, "source": "```{glue:figure} figure_corr_cbma\n:name: figure_corr_cbma-notebook\n:align: center\n\nAn array of plots of the corrected statistical maps produced by the different multiple comparisons correction methods.\n```" } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.10" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "state": { "01582731a18d4d0a9d5c9bbce9f87098": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "0811c6b0e9524003bac0047662c72819": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_bc371725951d48ce8234829ee8572c47", "placeholder": "​", "style": "IPY_MODEL_01582731a18d4d0a9d5c9bbce9f87098", "value": " 5000/5000 [1:49:27<00:00, 1.31it/s]" } }, "48f803e9c7784e49ab132b309f9a55c8": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "49cb3eacc7884a2aa1757551cc1f007d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "4d5d049c43b7484ea84f68d2d7cd7d6a": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_5147135c1df14281aa336b52a51773bf", "placeholder": "​", "style": "IPY_MODEL_49cb3eacc7884a2aa1757551cc1f007d", "value": "100%" } }, "5147135c1df14281aa336b52a51773bf": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "a00e84b502ac4a67ba23ea68263d9caa": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "a501a63690054e9185fc5cfa33d6d211": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_4d5d049c43b7484ea84f68d2d7cd7d6a", "IPY_MODEL_b41ad014f5e644ccb4d0578cd666d0a2", "IPY_MODEL_0811c6b0e9524003bac0047662c72819" ], "layout": "IPY_MODEL_edd52e48c769469fab8e03b82321f831" } }, "b41ad014f5e644ccb4d0578cd666d0a2": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_48f803e9c7784e49ab132b309f9a55c8", "max": 5000.0, "min": 0.0, "orientation": "horizontal", "style": "IPY_MODEL_a00e84b502ac4a67ba23ea68263d9caa", "value": 5000.0 } }, "bc371725951d48ce8234829ee8572c47": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "edd52e48c769469fab8e03b82321f831": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } } }, "version_major": 2, "version_minor": 0 } } }, "nbformat": 4, "nbformat_minor": 5 }