MODEL EVALUATION AND ANALYSIS
The modEvA package includes many of the functions preliminarily published in the modTools blog for analysing species distribution models and evaluating their performance. Note that modelling functions previously included in modEvA were moved to package fuzzySim.
modEvA is available on CRAN, but is more frequently updated on R-Forge. Check out the package homepage for more information, including begginer tutorials.
References:
The use of functions included in modEvA should be cited as:
Barbosa A.M., Brown J.A., Jiménez-Valverde A. & Real R. (2014) modEvA – Model Evaluation and Analysis. R package, version -.-
[cite the actual version used, which you can obtain in R with citation(package = “modEvA”)]
Please check out the package homepage for more references. If you use/cite modEvA functions, or find articles that do, please let me know so I can list them there and help justify the work dedicated to developing the package!
modTools 
Pingback: Plot predicted values for presences vs. absences | modTools
Pingback: Plot the density of predicted values for presences and absences | modTools
Can this package be used for calculating D-squared for a glm (generalized linear model in R) with gamma distribution?
Sorry, I’m not a stats expert, nor have experience with gamma-distribution models. In principle, the functions in modEvA will work if you provide them with a glm object containing the elements necessary for calculating the given measure (e.g. deviance, null.deviance). But the package was primarily designed with binomial GLMs in mind, and you’d need to check with a statistician to make sure that they are directly applicable to other GLM families. Let me know if you find anything that I should and could easily change in the package! Cheers
Hello,
I am having issues with the MillerCalib(model = mod) function.
I have a GLM which I have fit:
>mylogitMillerCalib(model = mod)
I get the following message:
Error: evaluation nested too deeply: infinite recursion / options(expressions=)?
Error during wrapup: evaluation nested too deeply: infinite recursion / options(expressions=)?
I have tried:
> options(expressions=10000)
But I am still getting the same message.
Any suggestions?
Many thanks!!
this is odd, I never got that error before… are you sure your ‘mod’ is a proper binomial glm object, as defined in the MillerCalib help file? and are you using the MillerCalib function, or something else called “>mylogitMillerCalib(model = mod)”?
Hi, i am trying to use varPart function with a lm model and three groups of variables, climate, topography and landscape heterogeneity. The group climate is formed by four variables and topography and landscape heterogeneity just for one. According to the varPart help the arguments ‘A’, ‘B’, ‘C’, ‘AB’, ‘AC’, ‘BC’ and ‘ABC’ are the numeric value of the R-squared of the regression of the response variables related to each factor. My question arises related to the R-squared value, because as in my case climate is formed by four variables, it is known that the the R-squeared value increased with incrising the number of variables. So in that case should i used the R-squared or the the adjusted R-squared, although is not specified in the help? Because my result chaged depending on the type of R used.
Additionally, i have read in the previous comments that the sum of the resulting values in the variance partitioning should sum 1. In my case i only get 1 if i write as input value of ‘ABC’ the value 1. If i used the R-squared value of the model considering the three factors (that is the R-squared of the original model) the sum of the resulting values is the ‘ABC’ R-squared value, that is the proportion related to this R-squared value. So i don’t know if this is ok, or i am doing something wrong.
The comand would be this
varPart(A = 0.404, B = 1.548e-08, C = 0.0105, AB = 0.5505, BC = 0.404, AC = 0.0105, ABC= 0.556, model.type = “LM”, A.name = “Climatic”, B.name = “topography”, C.name = “Landscape heterogeneity”, plot=TRUE, plot.digits = 3, cex.names = 1.2, cex.values = 1)
Thank you in advance!
Use the R-squared (non adjusted), because that is the actual amount of variation explained by your climatic model, so it can be summed to those of the other factors.
‘ABC’ must be (as defined in the ‘varPart’ help file) the R-squared of the regression of all variables from the 3 factors (i.e., the R-squared of the complete model); and the values in the circles should indeed sum up to ABC, which is the variation that they all explain in combination.
What should sum up to 1 is the result given in the R console, which (besides being more exact, i.e. less rounded) includes also the proportion of unexplained variation (when available, i.e. in the case of LMs). In the previous comments we were talking about GLMs, for which no (true) R-squared value is available, so there’s no ABC; in that case, the circles do sum up to 1, but they don’t represent proportions of explained variation of the response variable, but rather proportional contributions to the total variation explained by the model (see Details in ‘varPart’ help). But in your case (LM), there are true proportions of variation explained by each factor and the total model, and those are the ones to use.
I’m releasing a new version of package ‘modEvA’ (1.2.7, which should be available on R-Forge from tomorrow) in which the plot produced by ‘varPart’ also includes unexplained variation, and in which the help file has more examples and is hopefully a bit clearer, although these things are not easy to explain…
Hi, I’ve just tried the varPart and I think when model type = “GLM” the function doesn’t work properly. The problem is that each model r-square should be divided by the the full model r-squared but they are not. For exemple, the following line of code “Apure <- totalexpl – AB" should be "Apure <- totalexpl – BC/ABC". Same for the other lines. Currently the function give non-sensical result. It gives negative proportion and the portions do not sum to 1.
The line in the function is not “Apure <- totalexpl – AB", but rather "Apure <- totalexpl – BC". And is your model necessarily "GLM" (generalized linear) rather than "LM" (linear)? If it is, then there's no "full model r-squared", and you should input values as specified in the 2nd paragraph of Details in help(varPart); if it is a LM and you do input R-squared values, then you must use model.type = "LM". Negative proportions are not uncommon in variation partitioning, they just mean that two factors explain variance in opposite directions; but if the values do not sum up to 1, there must be a problem with your input values, I think.
Sorry, for the typo, I meant to say “Apure <- totalexpl – BC" should be "Apure <- totalexpl – BC/ABC" . Tell me if I'm wrong. The proportion of the full model (ABC) explained variance that is uniquely explained by A is : (ABC – BC)/ABC = 1 – BC/ABC.
I’m afraid you’re misinterpreting the way that A, B, AB etc. are defined, and not inputting the arguments as specified in help(varPart). These components already represent proportions of the total variation, so they don’t need to be divided by ABC.
If your model.type is GLM, then there’s no “model r-square” or “full model (ABC) explained variance”, as these models don’t have a coefficient of determination or total explained variance (which is a proportion, with a maximum of 1). What are you using as ABC? If you have a (true) R-squared for your total model, then you should specify model.type = “LM”.
Hi, I have just tried to install the package with this code:
install.packages(“modEvA”, repos=”http://R-Forge.R-project.org”)
Installing package into ‘/home/uoxyh/R/x86_64-pc-linux-gnu-library/library’
(as ‘lib’ is unspecified)
& this is the error that I received:
Warning in install.packages :
package ‘modEvA’ is not available (for R version 3.2.3)
Thank you.
Please follow the instructions in the second paragraph of section “Install and load” at http://modeva.r-forge.r-project.org/
how do you interpret the amount of deviance? What’s a good model, and what’s an acceptable range?
Sorry, not a stats expert… But basically, D-squared is calculated as (null deviance – residual deviance) / null deviance, i.e. how much deviance the given model explains the data more than a model with no variables. I’m not sure there are any objective threshold values for ‘good’ or ‘acceptable’ models, though.
I am not able to install the package from r-forge with the install command and the compressed files are not available for download, the build status states “Failed to build”
Is there some way around this issue?
Thank you
Katrina
hey,
I’ve tried to fix the problem on R-Forge, please try again tomorrow after the nightly rebuild of the packages there. Alternatively, a .tar.gz version of the package is (temporarily) available at https://www.dropbox.com/s/mc3gp6y8yew646n/modEvA_0.7.tar.gz?dl=0. Let me know if you have more trouble.
I’m getting an error message when I try to install: “package ‘modEvA’ is not available (for R version 3.0.3)”. Just FYI.
I was table to install using the devtool package’s install_url() function.
However, on OS X 10.9, that failed too initially because the R sets the environment variable TAR incorrectly. Sys.getenv(“TAR”) returns “usr/bin/gnutar”.
I added Sys.setenv(TAR = “usr/bin/tar”) to the .First() function in my .Rprofile file (which otherwise just looks like the one suggested on the Devtools GitHub page at https://github.com/hadley/devtools) and then I could install the package with install_url(“http://download.r-forge.r-project.org/src/contrib/modEvA_0.2.tar.gz”).
thanks – I’ve edited the page above accordingly