Do all the individual stars that we can see in the night sky belong to Milky Way? I was wondering if all the stars that we can see with the unaided eye as distinct point sources are from our own galaxy?
In other words, can we see stars from the Andromeda Galaxy or other galaxies without telescopes?
 A: Yes, but in January, 2015 the Hubble took photos of individual stars in the Andromeda galaxy.  But with the naked eye - no.
A: Yes, everything that appears as a point like star is in the Milky Way. The most nearby stars outside of the Milky Way are in the dwarf galaxies that are Milky Way satellites, such as the Large and Small Magellanic Clouds. These appear as fuzzy little blobs to the naked eye, just as Andromeda does. 
The only exception to this that I can think of is when a supernova occurs in a nearby galaxy. The most recent supernova visible to the naked eye was 1987A, which occurred in the Large Magellanic Cloud. Supernovae in Andromeda could also be visible to the naked eye as point sources. 
A: All point-objects in the sky are various entities in the Solar system (planets, etc.), or stars in our galaxy - with one exception:
http://en.wikipedia.org/wiki/Omega_Centauri
To the layperson, Omega Centauri looks point-like and is regularly thought of as a normal star. In fact, it's an extra-galactic star cluster; or, according to other opinions, a small galaxy remnant (depends on whether it has a central black hole or not). If you point a small telescope at it, it becomes obvious it's not a star.
Other than that, very rarely, a supernova in a nearby galaxy may become visible to the naked eye, but those don't last long.
TLDR: For the most part, yes, almost all those dots of light reside in our galaxy.
A: Yes all the stars that you can possibly observe by naked eye are from our own milky way galaxy.
You can not observe individual stars of another galaxy.
