Arguably the most important piece of the evidence-based-medicine puzzle is when we ask ourselves:
"Is this evidence significant? – Is this relevant to my patients and my practice?"
When we talk about the 'quality' of a published research work we largely mean what the epidemiologists refer to as 'internal validity' – the extent to which the study's conclusions are actually warranted given the methodology and results. Internal validity looks only at the study design, conduct and interpretation, and takes into account bias and confounders. While important, internal validity is not alone sufficient.
The significance of a piece of evidence to medicine in general, along with it's relevance to our own practice, is referred to as the external validity. I think that for your and my practice this is often what matters most.
Really, external validity just describes how well the results and conclusions can be generalized to situations and people beyond those in the study.
I think of significance as the cumulative generalizability of a piece of evidence for the specialty and for wider medicine, integrated with how well the evidence agrees with what is already known. Relevance describes how applicable the evidence is to my hospital, my practice – and my patients.
It has significance for you, and relevance for me.