My bank happens to belong to the Allpoint ATM network. Generally in England Allpoint ATM machines are outside Co-Op grocery stores among other businesses, which I looked up using my bank's website. My debit card also worked in other bank ATMs (e.g. Barclay's); I was charged a fee in that case by the owning bank and my bank.
Just now I tried Capital One's website searching for in-network ATMs in England, and I didn't have much luck. If you have their card already, did an information pamphlet come with it? The search screen on Capital One's website includes the Allpoint logo, and yet I couldn't get search results for within the UK.
If you don't have the Capital One card yet, I would question them about their network availability. If you bank somewhere else, your bank might very well belong to an international ATM network as does mine.