We do give you a Scripture reference for each story in LHTH, so you can read it from your own Bible of choice. However, I will share that if you can only buy one additional part of LHTH, I would choose The New Bible in Pictures for Little Eyes. My boys have all loved that Bible and considered it to be their very own. Each of my boys ended up having to have their own, as the covers were worn off of them.
The devotional comes up once each week and is easier to do without than the Bible. The Singing Bible is wonderful and does match the actual Bible story you are reading for that day, but again it is less important (in my opinion) than the Bible.
Anyway, other moms may feel differently and may be along to chime in with their opinion.