Is there a wasp nest in my roof?

My bedroom is the loft room of our house, and I've started hearing what sounds like scraping noises in the walls. I've also started seeing a lot of wasps flying past my bedroom window and also getting into my bedroom, which terrifies me coz im allergic to wasp stings. Is it the wasps making the scraping noises in the walls? Are they trying to bust in on me or something? I can't go and check anything out with my room being the loft...i've looked around my room and i dont see a nest, so if it is, have they got into the actual roof? which is completely inaccessible...so if they have how do i get rid? Or shall i just leave them be? :(