Hi Jen,
I also assembled my first desktop quite recently and I'm running Linux in it. My target specs were significantly higher than yours, but the cost of all the parts was around $650. If you want, I can give you the listing of want I built as a guide and you just remove or replace each part with something that fits your target goal. (Note that I'm living in the US; the prices and availability in your area may be different.)
Note that an alternative to assembling your own is to buy a barebone or small-factor PC from Newegg (under "Computer hardware"). These computers are usually sold without memory and disk, which is good because you can just add exactly what you want. The downside is that the CPUs are usually weaker or clocked down in order to fit into the small package and not run too hot. Another element to consider is that they might only support one output monitor.
If you are assembling your own, you should decide which parts you will need and make sure that you have the appropriate connectors between them. Also, you should look at different places for the best prices on each part. I ended up buying parts some parts from MicroCenter and some from Newegg.
In order to get the small form factor, you will most likely have to use a micro ATX or mini ITX motherboard. If you have a MicroCenter close to you, that's probably the best place to buy the motherboard and CPU. If you buy it as a bundle, you can get really good deals. I bought the AMD FX6300 with the ASRock Extreme and payed $170+tax for the bundle. (But, make sure to double-check all the prices at the register. In my case, the bundle discount was not applied automatically, for example.) You can get the same CPU with a micro ATX motherboard for $110+tax, and that's plenty of CPU... You will also need a case (make sure it supports the size of your motherboard), disk, memory, and maybe a graphics card (see below) and power supply. Note that if you buy a small factor case, some already have an integrated power supply, so check for that.
The downside of the AMD/motherboard configuration I mentioned above is that you would have to add a graphics card. There are two solutions: you select a CPU with an integrated GPU, or add a graphics card. The "fusion" CPUs from AMD have integrated GPUs that are comparable to those of entry-level graphics cards but they are poorly supported in Linux, so you end up with basically the same thing as the integrated graphics on an Intel CPU. Alternatively, you can get basic graphics cards from Newegg for $30. For example, an Nvidia GeForce 210 costs $30 and has 3 outputs (note that the graphics card requires a power supply of at least 300W, which may be a problem with some of small cases). I'm suggesting Nvidia because it currently has much better driver support in Linux.
Finally, note that although I pointed out AMD CPUs I'm not suggesting that they are better. I did so because, for the uses that you mentioned, they are cheaper and should provide more than enough compute power for several years.