RJafroc
Purpose
- This website contains four repositories describing observer performance and artificial intelligence systems modeling, analysis and validation.
- They are:
-
RJafroc
is the software package that provides the common thread on which the other three packages depend; -
RJafrocQuickStart
is an online book for those already somewhat familiar running Windows JAFROC. The obsolete Windows program has been replaced byRJafroc
. This book dives into how to useRJafroc
to analyze ROC or FROC datasets. -
RJafrocRocBook
is an online book providing necessary background on the ROC paradigm. -
RJafrocFrocBook
is an online book providing a detailed exposition of the FROC paradigm, particularly my contributions.
-
The first CRAN-posted version of RJafroc
was used to support the R-code examples in the book: Chakraborty DP: Observer Performance Methods for Diagnostic Imaging - Foundations, Modeling, and Applications with R-Based Examples, Taylor-Francis LLC, 2017. Since then the software has evolved considerably necessitating changes to the R-examples in the book. This, and other considerations, led me to conclude that an update to the book is needed. This website contains the updated software and three online books that use the software and extend the 2017-book.
Documentation
-
HTML documentation for
RJafroc
(functions, vignettes and update history) is available here. The software is fairly stable and undergoing occasional updates. -
The HTML online book
RJafrocQuickStart
is available here. A link is provided there to download the pdf book. Both are being continually updated (as of December 2021). -
The HTML online book
RJafrocRocBook
is available here. A link is provided there to download the pdf book. Both of them are being continually updated. -
The HTML online book
RJafrocFrocBook
is available here. A link is provided there to download the pdf book. Both are being continually updated.
Applications
While most of the applications in this package are geared toward analyzing radiologist performance in search tasks such as finding lesions in medical images, the software applies to any task involving detection and localization of targets in images. For example, the functions in RJafroc
can be used to analyze the performance of artificial intelligence (AI) algorithms. Two applications to AI are here, specifically:
- Measuring AI performance.
- Optimizing the reporting threshold of an AI algorithm.
The radiological search model (RSM), described here is implemented in RJafroc
. A fitting function RJafroc::FitRsmRoc
estimates RSM parameters from ROC data These parameters are related to search and classification performances:
- Search performance refers to finding lesions while simultaneously minimizing finding non-lesion locations
- Classification performance measures ability to distinguish between lesion and non-lesion locations.
Knowing the individual performances allows principled optimization of reader or AI algorithm performance.
Relation to Windows software
-
RJafroc
extends WindowsJAFROC
software and runs on multiple platforms. - Originally uploaded in 2004, the Windows software is many generations behind the software available on this website. However, many users find it to be easy to use and useful. Vignettes in
RJafroc
and the online book available here should allow one to quickly transition toRJafroc
. - If you still need Windows
JAFROC
software it is still available here.
Update History
- See here for details.
- Current
GitHub
version is 2.1.0.9000 on themaster
branch. - Current
CRAN
version is 2.1.0.
RJafroc
branch designations
- The most current tested version of the software and documentation is on the
master
branch. - The
developer
branch is for development/experimental work.
Installation: those already familiar with installing R packages from GitHub can ignore the following directions
How do I get set up?
Three options are given below in increasing order of complexity.
+ The first method downloads the package from CRAN.
+ The second downloads the package from the master
branch on GitHub
.
+ The third downloads all source files from the master
branch on GitHub
and then installs the software. I recommend the second method as the CRAN package is behind the master
branch.
1. Install from CRAN
- Install
R
andRStudio
. - Create an empty directory, e.g.,
myProject
. In my computer it is/Users/Dev/Downloads/myProject
. - Open
RStudio
by clicking on themyProject.Rproj
file. - Starting from
RStudio
>Packages
>Install
>RJafroc
. -
library(RJafroc)
. - This loads the CRAN package and all functions on the
cran2
branch become available. - For documentation of functions and vignettes, see. It may be helpful to have both windows open (
RStudio
and the above website) simultaneously. - The CRAN version will not include more recent updates on the
master
branch. To access them use the next suggested method, below. - Test the installation:
cbmPlot <- PlotCbmFit(c(1, 2), c(0.5, 0.5))
print(cbmPlot)
- You should see two ROC plots in the
Plots
window.
devtools
2. Install from GitHub using package - Install
R
andRStudio
. - Create an empty directory, e.g.,
myProject
. In my computer it is/Users/Dev/Downloads/myProject
. - Open
RStudio
by clicking on themyProject.Rproj
file. - Starting from
RStudio
>File
>New Project
>Existing Directory
> SelectmyProject
>Create Project
. - Oila! You should see
myProject.RProj
in the Files menu. - Install the
devtools
package as shown below: - Starting from
RStudio
>Packages
>Install
>devtools
. - Load
devtools
as shown below:
library(devtools)
- Install
RJafroc
directly fromGitHub
(this is wheredevtools
is used):
install_github("dpc10ster/RJafroc")
- Hit Enter on any prompts...
- Lots of activity and compilation of C++ code ....
- Load
RJafroc
as shown below:
library(RJafroc)
- Test the installation:
cbmPlot <- PlotCbmFit(c(1, 2), c(0.5, 0.5))
print(cbmPlot)
- You should see two ROC plots in the
Plots
window. - Documentation (vignettes) is available.
- Be sure to study these examples and make full use of the online documentation.
- Put your data and other files, if any, in
myProject
.
RJafroc
source files and install from the downloaded files
3. Download the - Clone this repository to a directory anywhere on your computer. On my computer it is in
/Users/Dev/Downloads/RJafroc
. Rename the folder if necessary to match my example. - I find the GitHub desktop app useful in mananging my downloads/uploads from
Git
. - Install
R
andRStudio
. - Navigate to the
RJafroc
directory. - Open
RJafroc.Rproj
. This will openRStudio
. - Navigate to
File
menu (lower-right window) and click on DESCRIPTION file. - Install all packages listed under Imports, e.g.,
openxlsx, ggplot2, stringr, tools, utils, stats, bbmle, binom, mvtnorm, dplyr, numDeriv, Rcpp - For example, to install the first two above-listed packages, use the following command at the Console prompt:
install.packages(c("openxlsx", "ggplot2"))
- Click on Build > Install and Restart (upper right panel). If errors result from missing packages, install those packages.
- A successful Install and Restart will result in the following line in the Console window:
library(RJafroc)
- Thats it!
RJafroc
has been installed to your computer and is visible to any otherR
project in any directory. - You will not need to access the
RJafroc
folder again (unless you reinstall a new version of the software). - Do not write any of your files to the
RJafroc
directory! - All necessary files of the installation are in a hidden directory that you do not normally need to worry about.
- Create an empty directory, e.g.,
myProject
, outside of theRJafroc
folder. In my computer it is/Users/Dev/Downloads/myProject
. - Starting from
RStudio
>File
>New Project
>Existing Directory
>myProject
>Create Project
. - Oila! You should see
myProject.RProj
in the Files menu. - Click on
Packages
and scroll down to findRJafroc
, and check the box next to it. This results inRJafroc
being loaded to the current workspace. The following line appears in the Console window (this is the hidden directory referrred to above).
library("RJafroc", lib.loc="/Library/Frameworks/R.framework/Versions/3.5/Resources/library")
- Click on
RJafroc
in the packages window. A help window opens up. I find it convenient to put this in its own window by clicking the "out" arrow button (hover message: Show in new window). You can access all documentation from here. - Test the installation:
cbmPlot <- PlotCbmFit(c(1, 2), c(0.5, 0.5))
print(cbmPlot)
- You should see two ROC plots in the
Plots
window. - Preliminary documentation (vignettes) is available.
- Put your data and other files, if any, in myProject.
- TBA
devtools
GitHub
page.)
Contibutor guidelines (adapted from -
As contributors and maintainers of this project, we pledge to respect all people who contribute through reporting issues, posting feature requests, updating documentation, submitting pull requests or patches, and other activities.
-
We are committed to making participation in this project a harassment-free experience for everyone, regardless of level of experience, gender, gender identity and expression, sexual orientation, disability, personal appearance, body size, race, ethnicity, age, or religion.
-
Examples of unacceptable behavior by participants include the use of sexual language or imagery, derogatory comments or personal attacks, trolling, public or private harassment, insults, or other unprofessional conduct.
-
Project maintainers have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct. Project maintainers who do not follow the Code of Conduct may be removed from the project team.
-
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported by opening an issue or contacting one or more of the project maintainers.