Can I can see both in stock and out of stock products at once?

Yes, this is definitely something you can do! All you need to do is remove the 'in stock/out of stock' filter completely. Just hover over the filter and click on the 'x' that appears.