i'm needing to set up a DSL modem, to which the computers connect through a router. all the computers are running winXP. is it possible to configure only certain computers to be allowed to access the internet, with password prompt? how so, or where can i find a tutorial on this?

You can limit other computers to have internet access but it is impossilbe to prompt them for password unless you have a domain controller and setup the computer policy but these will be more complicated.

If you are using a router it should be connected to wan port, so all computer connected to it are capable in accessing the internet but still depend on the configuration of each computers, if you want remove internet connection to some other computer you should configure the IP and the GATEWAY, including the DNS setup. remove the gateway to the specific computer and it will not be able to connect to internet.

I will do search for additional info for this and post it here as long as I found useful one...

um, yeah, the password thing i guess is kind of redundant. more applicable would be to have one main computer that can control the other's access. thanks for your help, and i look forward to more of your posts.

Your best bet there would be a linux router. The other computers will connect through that router to the net. However its not as easy as this, your modem needs to be able to relay DNS and gateway information, you'll need multiple network cards etc...
If you only want to disable some computers then the easiest way is to block their MAC addresses which most modems and routers can do.