Experimental Design - Revision history

Wordpad: BloomWiki: Experimental Design

2026-04-25T01:50:54Z

BloomWiki: Experimental Design

← Older revision		Revision as of 01:50, 25 April 2026
Line 1:		Line 1:
			<div style="background-color: #4B0082; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
	{{BloomIntro}}		{{BloomIntro}}
	Experimental Design is the process of planning an experiment to reach objective and valid conclusions efficiently. It is the "blueprint" of science. Whether testing a new drug, a marketing campaign, or a physical theory, a well-designed experiment ensures that the results are due to the factor being studied and not to chance or "noise." Experimental design involves determining the variables to be measured, the number of participants needed (power), and the methods for reducing bias (like randomization and blinding). It is the bridge between a hypothesis and a discovery.		Experimental Design is the process of planning an experiment to reach objective and valid conclusions efficiently. It is the "blueprint" of science. Whether testing a new drug, a marketing campaign, or a physical theory, a well-designed experiment ensures that the results are due to the factor being studied and not to chance or "noise." Experimental design involves determining the variables to be measured, the number of participants needed (power), and the methods for reducing bias (like randomization and blinding). It is the bridge between a hypothesis and a discovery.
			</div>

	== Remembering ==		__TOC__

			<div style="background-color: #000080; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
			== <span style="color: #FFFFFF;">Remembering</span> ==
	* '''Experimental Design''' — The process of planning a study to meet specified objectives.		* '''Experimental Design''' — The process of planning a study to meet specified objectives.
	* '''Independent Variable (IV)''' — The factor that is manipulated by the researcher (the "cause").		* '''Independent Variable (IV)''' — The factor that is manipulated by the researcher (the "cause").
Line 18:		Line 23:
	* '''Internal Validity''' — The degree to which the experiment shows a true cause-and-effect relationship.		* '''Internal Validity''' — The degree to which the experiment shows a true cause-and-effect relationship.
	* '''External Validity''' — The degree to which the results can be generalized to other settings or populations.		* '''External Validity''' — The degree to which the results can be generalized to other settings or populations.
			</div>

	== Understanding ==		<div style="background-color: #006400; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
			== <span style="color: #FFFFFF;">Understanding</span> ==
	A good experiment is built on four main principles (Fisher's Principles).		A good experiment is built on four main principles (Fisher's Principles).

Line 31:		Line 38:

	'''The Power Problem''': If your sample size is too small, your experiment might fail to find an effect even if it exists (a "False Negative" or Type II error). Experimental designers use '''Power Analysis''' to calculate the minimum number of people needed ''before'' the experiment starts.		'''The Power Problem''': If your sample size is too small, your experiment might fail to find an effect even if it exists (a "False Negative" or Type II error). Experimental designers use '''Power Analysis''' to calculate the minimum number of people needed ''before'' the experiment starts.
			</div>

	== Applying ==		<div style="background-color: #8B0000; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
			== <span style="color: #FFFFFF;">Applying</span> ==
	'''Modeling the 'Placebo Effect' and 'Treatment Effect':'''		'''Modeling the 'Placebo Effect' and 'Treatment Effect':'''
	<syntaxhighlight lang="python">		<syntaxhighlight lang="python">
Line 66:		Line 75:
	: '''Natural Experiment''' → Using a "random" event in the real world (like a lottery or a sudden policy change) as if it were an experiment.		: '''Natural Experiment''' → Using a "random" event in the real world (like a lottery or a sudden policy change) as if it were an experiment.
	: '''A/B/n Testing''' → Testing multiple versions (A, B, C, D) of a product at once.		: '''A/B/n Testing''' → Testing multiple versions (A, B, C, D) of a product at once.
			</div>

	== Analyzing ==		<div style="background-color: #8B4500; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
			== <span style="color: #FFFFFF;">Analyzing</span> ==
	{\| class="wikitable"		{\| class="wikitable"
	\|+ Internal vs. External Validity		\|+ Internal vs. External Validity
Line 78:		Line 89:

	'''The Replication Crisis''': In many fields (especially psychology and medicine), researchers have found that they cannot replicate the results of famous experiments. This is often due to poor experimental design, such as '''P-hacking''' (testing many things until something looks "significant") or having very low '''Statistical Power'''. Modern experimental design emphasizes '''Pre-registration''' (publicly stating your plan before you start) to ensure honesty.		'''The Replication Crisis''': In many fields (especially psychology and medicine), researchers have found that they cannot replicate the results of famous experiments. This is often due to poor experimental design, such as '''P-hacking''' (testing many things until something looks "significant") or having very low '''Statistical Power'''. Modern experimental design emphasizes '''Pre-registration''' (publicly stating your plan before you start) to ensure honesty.
			</div>

	== Evaluating ==		<div style="background-color: #483D8B; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
			== <span style="color: #FFFFFF;">Evaluating</span> ==
	Evaluating an experimental design:		Evaluating an experimental design:
	# '''Feasibility''': Can we actually afford the sample size needed for high power?		# '''Feasibility''': Can we actually afford the sample size needed for high power?
Line 85:		Line 98:
	# '''Attrition''': What happens if 20% of the participants drop out halfway through?		# '''Attrition''': What happens if 20% of the participants drop out halfway through?
	# '''Sensitivity''': Is the measurement tool (e.g., a survey or a blood test) accurate enough to detect the effect?		# '''Sensitivity''': Is the measurement tool (e.g., a survey or a blood test) accurate enough to detect the effect?
			</div>

	== Creating ==		<div style="background-color: #2F4F4F; color: #FFFFFF; padding: 20px; border-radius: 8px; margin-bottom: 15px;">
			== <span style="color: #FFFFFF;">Creating</span> ==
	Future Frontiers:		Future Frontiers:
	# '''Adaptive Designs''': Experiments that change their "rules" (e.g., sample size or dosage) in real-time as data comes in to reach a conclusion faster.		# '''Adaptive Designs''': Experiments that change their "rules" (e.g., sample size or dosage) in real-time as data comes in to reach a conclusion faster.
Line 96:		Line 111:
	[[Category:Science]]		[[Category:Science]]
	[[Category:Research Methods]]		[[Category:Research Methods]]
			</div>

Wordpad: BloomWiki: Experimental Design

2026-04-23T14:37:37Z

BloomWiki: Experimental Design

← Older revision		Revision as of 14:37, 23 April 2026
Line 80:		Line 80:

	== Evaluating ==		== Evaluating ==
	Evaluating an experimental design: ~~(1)~~ '''Feasibility''': Can we actually afford the sample size needed for high power? ~~(2)~~ '''Ethics''': Is it okay to withhold a potentially life-saving treatment from the control group? ~~(3)~~ '''Attrition''': What happens if 20% of the participants drop out halfway through? ~~(4)~~ '''Sensitivity''': Is the measurement tool (e.g., a survey or a blood test) accurate enough to detect the effect?		Evaluating an experimental design:
			# '''Feasibility''': Can we actually afford the sample size needed for high power?
			# '''Ethics''': Is it okay to withhold a potentially life-saving treatment from the control group?
			# '''Attrition''': What happens if 20% of the participants drop out halfway through?
			# '''Sensitivity''': Is the measurement tool (e.g., a survey or a blood test) accurate enough to detect the effect?

	== Creating ==		== Creating ==
	Future Frontiers: ~~(1)~~ '''Adaptive Designs''': Experiments that change their "rules" (e.g., sample size or dosage) in real-time as data comes in to reach a conclusion faster. ~~(2)~~ '''In Silico Experiments''': Using massive computer simulations of the human body or the climate to run experiments that would be impossible or unethical in real life. ~~(3)~~ '''Multi-Armed Bandits''': AI algorithms that "experiment" with different options and slowly shift resources toward the winning one. ~~(4)~~ '''Open Science Framework''': Moving toward a global culture where every experimental design and dataset is shared and verifiable by everyone.		Future Frontiers:
			# '''Adaptive Designs''': Experiments that change their "rules" (e.g., sample size or dosage) in real-time as data comes in to reach a conclusion faster.
			# '''In Silico Experiments''': Using massive computer simulations of the human body or the climate to run experiments that would be impossible or unethical in real life.
			# '''Multi-Armed Bandits''': AI algorithms that "experiment" with different options and slowly shift resources toward the winning one.
			# '''Open Science Framework''': Moving toward a global culture where every experimental design and dataset is shared and verifiable by everyone.

	[[Category:Statistics]]		[[Category:Statistics]]
	[[Category:Science]]		[[Category:Science]]
	[[Category:Research Methods]]		[[Category:Research Methods]]

Wordpad: BloomWiki: Experimental Design

2026-04-23T14:28:24Z

BloomWiki: Experimental Design

← Older revision		Revision as of 14:28, 23 April 2026
Line 22:		Line 22:
	A good experiment is built on four main principles (Fisher's Principles).		A good experiment is built on four main principles (Fisher's Principles).

	'''1. Comparison''': You must have a "Control." If you give 10 people a cold medicine and they get better in 5 days, you haven't proven anything unless you know how long it takes people without the medicine to get better.		'''1. Comparison''': You must have a "Control." If you give 10 people a cold medicine and they get better in 5 days, you haven't proven anything unless you know how long it takes people ''without'' the medicine to get better.

	'''2. Randomization''': This is the "magic" of experimental design. It ensures that any "hidden" differences (like genes, diet, or personality) are spread equally across both groups, so they "cancel out."		'''2. Randomization''': This is the "magic" of experimental design. It ensures that any "hidden" differences (like genes, diet, or personality) are spread equally across both groups, so they "cancel out."
Line 28:		Line 28:
	'''3. Replication''': One success could be luck. Doing the experiment many times (or with many people) reduces the impact of random "noise."		'''3. Replication''': One success could be luck. Doing the experiment many times (or with many people) reduces the impact of random "noise."

	'''4. Blocking (Grouping)''': If you know that age or gender will affect the result, you can "block" your participants into groups (e.g., Males and Females) and then randomize within those blocks. This increases the "precision" of the experiment.		'''4. Blocking (Grouping)''': If you know that age or gender will affect the result, you can "block" your participants into groups (e.g., Males and Females) and then randomize ''within'' those blocks. This increases the "precision" of the experiment.

	'''The Power Problem''': If your sample size is too small, your experiment might fail to find an effect even if it exists (a "False Negative" or Type II error). Experimental designers use '''Power Analysis''' to calculate the minimum number of people needed before the experiment starts.		'''The Power Problem''': If your sample size is too small, your experiment might fail to find an effect even if it exists (a "False Negative" or Type II error). Experimental designers use '''Power Analysis''' to calculate the minimum number of people needed ''before'' the experiment starts.

	== Applying ==		== Applying ==

Wordpad: BloomWiki: Experimental Design

2026-04-23T14:22:27Z

BloomWiki: Experimental Design

← Older revision		Revision as of 14:22, 23 April 2026
Line 22:		Line 22:
	A good experiment is built on four main principles (Fisher's Principles).		A good experiment is built on four main principles (Fisher's Principles).

	1. Comparison: You must have a "Control." If you give 10 people a cold medicine and they get better in 5 days, you haven't proven anything unless you know how long it takes people without the medicine to get better.		'''1. Comparison''': You must have a "Control." If you give 10 people a cold medicine and they get better in 5 days, you haven't proven anything unless you know how long it takes people without the medicine to get better.

	2. Randomization: This is the "magic" of experimental design. It ensures that any "hidden" differences (like genes, diet, or personality) are spread equally across both groups, so they "cancel out."		'''2. Randomization''': This is the "magic" of experimental design. It ensures that any "hidden" differences (like genes, diet, or personality) are spread equally across both groups, so they "cancel out."

	3. Replication: One success could be luck. Doing the experiment many times (or with many people) reduces the impact of random "noise."		'''3. Replication''': One success could be luck. Doing the experiment many times (or with many people) reduces the impact of random "noise."

	4. Blocking (Grouping): If you know that age or gender will affect the result, you can "block" your participants into groups (e.g., Males and Females) and then randomize within those blocks. This increases the "precision" of the experiment.		'''4. Blocking (Grouping)''': If you know that age or gender will affect the result, you can "block" your participants into groups (e.g., Males and Females) and then randomize within those blocks. This increases the "precision" of the experiment.

	The Power Problem: If your sample size is too small, your experiment might fail to find an effect even if it exists (a "False Negative" or Type II error). Experimental designers use Power Analysis to calculate the minimum number of people needed before the experiment starts.		'''The Power Problem''': If your sample size is too small, your experiment might fail to find an effect even if it exists (a "False Negative" or Type II error). Experimental designers use '''Power Analysis''' to calculate the minimum number of people needed before the experiment starts.

	== Applying ==		== Applying ==
Line 77:		Line 77:
	\|}		\|}

	The Replication Crisis: In many fields (especially psychology and medicine), researchers have found that they cannot replicate the results of famous experiments. This is often due to poor experimental design, such as P-hacking (testing many things until something looks "significant") or having very low Statistical Power. Modern experimental design emphasizes Pre-registration (publicly stating your plan before you start) to ensure honesty.		'''The Replication Crisis''': In many fields (especially psychology and medicine), researchers have found that they cannot replicate the results of famous experiments. This is often due to poor experimental design, such as '''P-hacking''' (testing many things until something looks "significant") or having very low '''Statistical Power'''. Modern experimental design emphasizes '''Pre-registration''' (publicly stating your plan before you start) to ensure honesty.

	== Evaluating ==		== Evaluating ==
	Evaluating an experimental design: (1) Feasibility: Can we actually afford the sample size needed for high power? (2) Ethics: Is it okay to withhold a potentially life-saving treatment from the control group? (3) Attrition: What happens if 20% of the participants drop out halfway through? (4) Sensitivity: Is the measurement tool (e.g., a survey or a blood test) accurate enough to detect the effect?		Evaluating an experimental design: (1) '''Feasibility''': Can we actually afford the sample size needed for high power? (2) '''Ethics''': Is it okay to withhold a potentially life-saving treatment from the control group? (3) '''Attrition''': What happens if 20% of the participants drop out halfway through? (4) '''Sensitivity''': Is the measurement tool (e.g., a survey or a blood test) accurate enough to detect the effect?

	== Creating ==		== Creating ==
	Future Frontiers: (1) Adaptive Designs: Experiments that change their "rules" (e.g., sample size or dosage) in real-time as data comes in to reach a conclusion faster. (2) In Silico Experiments: Using massive computer simulations of the human body or the climate to run experiments that would be impossible or unethical in real life. (3) Multi-Armed Bandits: AI algorithms that "experiment" with different options and slowly shift resources toward the winning one. (4) Open Science Framework: Moving toward a global culture where every experimental design and dataset is shared and verifiable by everyone.		Future Frontiers: (1) '''Adaptive Designs''': Experiments that change their "rules" (e.g., sample size or dosage) in real-time as data comes in to reach a conclusion faster. (2) '''In Silico Experiments''': Using massive computer simulations of the human body or the climate to run experiments that would be impossible or unethical in real life. (3) '''Multi-Armed Bandits''': AI algorithms that "experiment" with different options and slowly shift resources toward the winning one. (4) '''Open Science Framework''': Moving toward a global culture where every experimental design and dataset is shared and verifiable by everyone.

	[[Category:Statistics]]		[[Category:Statistics]]
	[[Category:Science]]		[[Category:Science]]
	[[Category:Research Methods]]		[[Category:Research Methods]]

Wordpad: BloomWiki: Experimental Design

2026-04-23T13:38:12Z

BloomWiki: Experimental Design

New page

{{BloomIntro}}
Experimental Design is the process of planning an experiment to reach objective and valid conclusions efficiently. It is the "blueprint" of science. Whether testing a new drug, a marketing campaign, or a physical theory, a well-designed experiment ensures that the results are due to the factor being studied and not to chance or "noise." Experimental design involves determining the variables to be measured, the number of participants needed (power), and the methods for reducing bias (like randomization and blinding). It is the bridge between a hypothesis and a discovery.

== Remembering ==
* '''Experimental Design''' — The process of planning a study to meet specified objectives.
* '''Independent Variable (IV)''' — The factor that is manipulated by the researcher (the "cause").
* '''Dependent Variable (DV)''' — The factor that is measured (the "effect").
* '''Control Group''' — The group that does not receive the treatment, used as a baseline.
* '''Experimental Group''' — The group that receives the treatment.
* '''Randomization''' — Assigning participants to groups by chance to ensure the groups are similar at the start.
* '''Blinding''' — Keeping the participants (Single-blind) or both participants and researchers (Double-blind) unaware of who is in which group.
* '''Placebo''' — An inactive substance or treatment given to the control group.
* '''Sample Size (N)''' — The number of observations or participants in an experiment.
* '''Statistical Power''' — The probability that an experiment will detect an effect if one actually exists.
* '''Confounding Variable''' — An outside influence that changes the effect of a dependent and independent variable.
* '''Factorial Design''' — An experiment that studies the effect of two or more independent variables simultaneously.
* '''Replication''' — Repeating an experiment to see if the same results are achieved.
* '''Internal Validity''' — The degree to which the experiment shows a true cause-and-effect relationship.
* '''External Validity''' — The degree to which the results can be generalized to other settings or populations.

== Understanding ==
A good experiment is built on four main principles (Fisher's Principles).

**1. Comparison**: You must have a "Control." If you give 10 people a cold medicine and they get better in 5 days, you haven't proven anything unless you know how long it takes people *without* the medicine to get better.

**2. Randomization**: This is the "magic" of experimental design. It ensures that any "hidden" differences (like genes, diet, or personality) are spread equally across both groups, so they "cancel out."

**3. Replication**: One success could be luck. Doing the experiment many times (or with many people) reduces the impact of random "noise."

**4. Blocking (Grouping)**: If you know that age or gender will affect the result, you can "block" your participants into groups (e.g., Males and Females) and then randomize *within* those blocks. This increases the "precision" of the experiment.

**The Power Problem**: If your sample size is too small, your experiment might fail to find an effect even if it exists (a "False Negative" or Type II error). Experimental designers use **Power Analysis** to calculate the minimum number of people needed *before* the experiment starts.

== Applying ==
'''Modeling the 'Placebo Effect' and 'Treatment Effect':'''
<syntaxhighlight lang="python">
import numpy as np

def run_experiment(n_per_group, true_effect_size):
"""
Simulates an experiment with a Control (Placebo)
and an Experimental (Treatment) group.
"""
# Placebo effect (everyone gets a bit better just by being in a study)
placebo_mean = 5

# Generate data with random noise
control = np.random.normal(placebo_mean, 2, n_per_group)
experimental = np.random.normal(placebo_mean + true_effect_size, 2, n_per_group)

# Calculate observed difference
observed_diff = np.mean(experimental) - np.mean(control)
return observed_diff

# Is the medicine effective? (Effect size = 3)
n = 50 # Sample size
observed = run_experiment(n, 3)
print(f"Observed Treatment Effect (N={n}): {observed:.2f}")
# If N is small, the 'noise' might make the effect look like 0 or 10.
# Increasing N 'averages out' the noise.
</syntaxhighlight>

; Design Types
: '''Cross-over Design''' → Each participant receives both treatments at different times (they act as their own control).
: '''Quasi-Experiment''' → An experiment where randomization is not possible (e.g., comparing two different classrooms).
: '''Natural Experiment''' → Using a "random" event in the real world (like a lottery or a sudden policy change) as if it were an experiment.
: '''A/B/n Testing''' → Testing multiple versions (A, B, C, D) of a product at once.

== Analyzing ==
{| class="wikitable"
|+ Internal vs. External Validity
! Type !! Definition !! Threat
|-
| Internal || Is the effect real? || Confounding variables, Bias, Bad measurement
|-
| External || Does it work elsewhere? || "Lab" environment being too different from the real world
|}

**The Replication Crisis**: In many fields (especially psychology and medicine), researchers have found that they cannot replicate the results of famous experiments. This is often due to poor experimental design, such as **P-hacking** (testing many things until something looks "significant") or having very low **Statistical Power**. Modern experimental design emphasizes **Pre-registration** (publicly stating your plan before you start) to ensure honesty.

== Evaluating ==
Evaluating an experimental design: (1) **Feasibility**: Can we actually afford the sample size needed for high power? (2) **Ethics**: Is it okay to withhold a potentially life-saving treatment from the control group? (3) **Attrition**: What happens if 20% of the participants drop out halfway through? (4) **Sensitivity**: Is the measurement tool (e.g., a survey or a blood test) accurate enough to detect the effect?

== Creating ==
Future Frontiers: (1) **Adaptive Designs**: Experiments that change their "rules" (e.g., sample size or dosage) in real-time as data comes in to reach a conclusion faster. (2) **In Silico Experiments**: Using massive computer simulations of the human body or the climate to run experiments that would be impossible or unethical in real life. (3) **Multi-Armed Bandits**: AI algorithms that "experiment" with different options and slowly shift resources toward the winning one. (4) **Open Science Framework**: Moving toward a global culture where every experimental design and dataset is shared and verifiable by everyone.

[[Category:Statistics]]
[[Category:Science]]
[[Category:Research Methods]]