I'm kinda rusty, but I'll see what I can glean from some old x86 emitters I wrote in NASM macros a few years ago.
I'm just going to talk about the basic instruction set for 80386 (ie, no extended instructions). Note that this assumes that the instructions are in protected mode.
First, you have prefix bytes. These can change the arithmetic or memory pointer word size from the current mode's to the other one (ie, if you're in protected mode you have a word size of 32bits, so using these prefixes can go to 16bits). They can do other things like repeat certain instructions, lock the coprocessor, or override the default segment register with another, but in protected mode only the word size changing ones are really useful, and these are:
0x66: Treat arithmetic operands as in opposite mode, IE if doing add eax, 12345 then it'll really use ax and the immediate will be 16bits long instead of 32.
0x67: Treat memory operands as in opposite mode, IE if doing add eax, [ebx + 12345] it'll use bx instead of ebx and a 16bit displacement instead of a 32bit one.
There can be zero or multiple prefix bytes. I don't know the exact limit, but I'm pretty sure the overall instruction cannot be more than 15 bytes.
Then you have the opcode. This is one or two bytes and says what instruction it is. Some opcodes also have an operand built in, these are special small form instructions.
If there are any operands not implicit in the instruction itself you then have a Mod R/M byte which specifies the operands. If there are two operands then at least one of them is always in a register. The top two bits specifies where the other operand (the non-register one) is.
0 means it's at a memory location with a register offset.
1 means it's at a memory location with a register + 8bit immediate offset.
2 means it's at a memory location with a register + 32bit immediate offset (or 16bits if prefix 0x67 is used)
3 means it's in a register.
Then the next lower 6bits specify the registers to use. The upper 3bits specify the always register value and use the standard register encoding which goes eax, ecx, edx, ebx, esp, ebp, esi, edi. The lower 3bits specify the memory base register, however, ebp and esp have special usage. ebp in non-displacement indexing means direct address with word size immediate, so a 16/32bit immediate is used as a memory address directly.
If it's esp then a special memory address mode is used, and the next byte is called the SIB, Scaled Index Byte. The SIB byte allows you to create a complex memory address that can add a base register, an index register multiplied by a scale value, and an offset together to form the address. The Mod R/M will determine if there's an immediate displacement or not.
The top two bits of the SIB byte determines the scale, it can be a value from 0 to 3 which means scale by 1, 2, 4, or 8. Of the next 6 bits, the top 3 specify the index register, and the bottom 3 the base register. Using esp as the index register really means no index register should be used (I think, I can't find this clearly in my source but I'm pretty sure this is how it works). The only reason you'd want to have no scaled index is so you can address relative to esp.
Next comes the 8, 16, or 32bit memory displacement as specified in the Mod R/M byte if it has one.
Finally, there's an 8, 16, or 32bit arithmetic immediate if the instruction has one (this is implicit to the opcode).
So if you want to decode an x86 instruction, first you should grab all the prefix bytes. Then you get the first byte of the opcode, and if it's a 2byte one get the second one. Then if the instruction has non-implicit non-immediate operands get the Mod R/M byte. If the address mode is memory and the base register is esp, get the SIB. If there's a memory displacement get it, and if there's an immediate operand get that.
If you're disassembling, determine the names of each of the prefix bytes, then the instruction; if the instruction has an implicit operand grab it, then determine the other operands (display them based on the address mode). I'd recommend looking up the names of things using a lookup table.
That should cover most of the basics, hope I didn't get anything wrong. Let me know if you have any questions. People complain about x86 a lot but if you ask me some aspects of the instruction encoding are more straightforward than say, ARM (and others are a lot worse..)