Are you interested in studying pediatric mental health? Help us analyze the Healthy Brain Network dataset!
We want to measure the volumes of various structures in the brain with automated algorithms (such as Klein et al., 2017, Fischl et al., 2002, and Avants et al., 2014). But some of the scans are very bad quality and would yield incorrect results. We need to identify these scans so we don't analyze them.
To begin, sign the consent form and create an account:
Next, read the instructions on how to view images and save your ratings:
Figure 1: An animation showing how to open and image and adjust brightness/contrast
1. Open an image
Click on an image in the "Image File" column of the table below. After the image loads you can adjust the brightness and contrast by right clicking and dragging, or adjusting the colormap on the top left.
2. Pass or Fail the image & rate your Confidence
In the "Save Changes" menu on the left side, click either "Pass" or "Fail". Then rate your confidence on a scale of 1 (low confidence) to 5 (high confidence). Write any observations in the "Notes" sections, and click "Save" when you are done.
Figure 2: The QC menu panel
Follow these guidelines when you rate the images:
Scans with too much motion cannot be segmented
The image below shows varying amounts of motion artifacts (you can see bands on the images). Images with too much motion cannot be automatically segmented because its too difficult to see the boundaries between tissues.
Figure 3: Four images with varying levels on motion (banding artifacts). Image on the left has many artifacts, and it is difficult to delineate tissue types. The red arrow on the second image from the left shows the banding artifact to a lesser degree, and the green arrow on the third image from the left points to very minor banding artifacts. Motion guidelines are adapted from (Walhovd et al., 2016)
If you are not sure how to rate a scan with motion, give your rating a low confidence score (a 1 or 2). Don't worry if you are wrong, we will be averaging ratings across many raters.
Shift + Click to annotate slices or regions of the image that are low quality
Sometimes artifacts are only on a few slices. Annotate these slices so we know where they are:
Figure 4: Three examples showing images with artifacts only on a few slices. The image on the right shows a defacing error in the frontal lobe. The circle on the center image highlights an abnormally noisy region, and the image on the left is noisy throughout the whole image.
The red dots on the image above show the annotations in regions of noise. Scroll through the entire brain when you QC, and watch out for bad defacing in the frontal lobes.
Automated segmentation might fail on abnormal geometry
The images below show brains with abnormal geometry, such as cysts or abnormally large ventricles. Automated segmentations might fail on these scans, so we need to be warned ahead of time.
Figure 5: Three examples of scans with good quality but abnormal geometry
Pass the image if its quality is good, but remember to write a note (in the notes section) about any abnormal geometry or incidental findings.
Rank | Username | # |
---|---|---|
{{idx}} | {{username}} | {{images}} |
Anisha Keshavan
University of Washington eScience Institute and Institute for Neuroengineering
keshavan@uw.edu
Jason Yeatman
University of Washington Institute for Neuroengineering
jdyeatman@uw.edu
Ariel Rokem
University of Washington eScience Institute
arokem@uw.edu
We are asking you to be in a research study. The purpose of this consent form is to give you the information you will need to help you decide whether to be in the study or not. Please read the form carefully. You may ask questions about the purpose of the research, what we would ask you to do, the possible risks and benefits, your rights as a volunteer, and anything else about the research or this form that is not clear. Please email keshavan@uw.edu if you have any questions. When we have answered all your questions, you can decide if you want to be in the study or not. This process is called “informed consent.”
Image annotation is needed to extract data from images. The purpose of this study is to train better computer algorithms to annotate images, and also correct any errors the computer makes. We aim to 1) collect image annotation data from you, and 2) combine annotations from many users into the most accurate annotation. This information will be used to train new computer algorithms.
You will be asked to log in. You may be given the option to use an existing social media login (e.g. Twitter, Facebook, Google, etc.), and you may be asked to create a an anonymous username to be displayed to the public. You may also choose to include your social media avatar to display alongside your anonymous username in the public leaderboard. Alternatively, you may also have the option to use an anonymous login. You may provide your email when you sign up for an account if you wish to be contacted about future work related to this task.
You will be given instructions on how to annotate images for a particular task. Try your best to annotate the images as quickly and accurately as you can. You may complete however many tasks you wish. Each task can take anywhere from 30 seconds to 5 minutes. You may stop annotating at any time.
You may use any device with a web browser (computer, tablet, or phone). We will record all settings you use on the application, your annotations, your screen resolution, and your anonymous username. This data will be made publically available to researchers.
Some people feel that providing information for research or having the research session recorded is an invasion of privacy. If you wish to remain anonymous, we recommend creating an anonymous nickname that does not include any identifiers relating to you.
This study will help us to extract accurate data from images. This information will be used for scientific research.
Your social media login information (such as username and email address), if provided, will remain confidential and will not be shared outside the study team. Government or university staff sometimes review studies such as this one to make sure they are being done safely and legally. If a review of this study takes place, your records may be examined. The reviewers will protect your privacy. The study records will not be used to put you at legal risk of harm.
If you think you have been harmed from being in this research, contact Anisha Keshavan at keshavan@uw.edu.
* Please note that we cannot ensure the confidentiality of information sent via e-mail.
This study has been explained to me, and I understand the procedures, benefits, and risks of this study. I affirm that I am capable of consenting on my own behalf, that I am age 18 or older, and I volunteer to take part in this research. I have had a chance to ask questions. If I have questions later about the research, or if I have been harmed by participating in this study, I can contact one of the researchers listed on the first page of this consent form. If I have questions about my rights as a research subject, I can call the Human Subjects Division at (206) 543-0098 or email hsdinfo@uw.edu.