What word would you use when the "belief" is not in doubt and/or is supported by evidence?
Wicki says, "Belief is the psychological state in which an individual holds a proposition or premise to be true." Under "belief" Wicki also links to a topic called "religious beliefs," but nothing there seems to support your definition?
dictionary.com says
be⋅lief
–noun
1. something believed; an opinion or conviction: a belief that the earth is flat.
2. confidence in the truth or existence of something not immediately susceptible to rigorous proof: a statement unworthy of belief.
3. confidence; faith; trust: a child's belief in his parents.
4. a religious tenet or tenets; religious creed or faith: the Christian belief.
I can find no definitive authority for your proposition. When I read the definitions from dictionary.com they lean your way. They leave one thinking that a belief is less concrete than a conviction. Still, if I believe I have my hat, must I only do so in the absence of evidence? I do not accept that use of the word "believe" or its companions such as "belief" necessarily concludes an element of doubt.
I do understand where you are coming from. I suspect that it depends on the context. Use of the word might often mean that the user acknowledges an element of doubt or that not everybody might agree. However it might also mean an assertion that something is not only true, but true in the experience of the speaker.