An identifier actually identifies something, it's not the thing being identified. The splitting of hairs in this case is important.
They're typically kept track of by the symbol table.
If you have a location in memory that holds a value that changes, that's a variable. You need a way to reference that "thing." The identifier is the name you use in your program which represents where that value is and what datatype it is of.
Note, you could just as well have a value that changes that the compiler holds entirely in a register and never goes into memory; you still need a way to access it (like incrementing a loop index or a counter). The identifier is the thing you call it in the program, and it knows to look in a register and how to treat that value, rather than going to memory.
You could have a value that doesn't change in protected memory; the identifier again indicates where the value is, what kind of data it is, and clues the compiler into not letting you change the value.
In fact the constant value doesn't even have to be in memory. The identifier can just substitute the value in place of the identifier, given the appropriate compiler optimizations.
It's important to realize the identifier isn't a pointer type. Often it's addresses of where something is stored in the heap, a position on the stack, a temporary register, a location in memory where there's a function you intend to call, a location in the program flow for a break or a dreaded goto statement, etc.
You might think of it as the "name" of something, or a "symbol" you assign to it.