Probably you are looking for a programmable microcontroller. If you have experience in C/C++/Java, checkout Arduino. Its chip is programmed using a C like language. This “How tos” page might help you get started. There are also some good books that will help you move forward:
Wiring is a platform similar to Arduino.
Also have a look at the Forth programming language. There are lot of interesting “tiny computers” that you can program with this rather unusual language. Here is a partial list:
Two famous Forth books:
- Starting Forth
- Thinking Forth (A classic in Software Engineering literature.)