The question is whether time is an operator in the sense of ${\hat T}|t\rangle~=~t|t\rangle$. This at first glance would seem to make sense because we do have a position operator ${\hat X}|x\rangle~=~x|x\rangle$. However, this does not work. This is a subtle question in many ways.
Quantum mechanics is unitary. Consider a state vector $|\psi(t)\rangle$ evolve into a small increment of time so $|\psi(t)\rangle~\rightarrow~|\psi(t + \delta t)\rangle$. A Taylor expansion this gives
$$
\psi(t + \delta t)\rangle~=~|\psi(t)\rangle + \frac{\partial|\psi(t)\rangle}{\partial t}\delta t + O(\delta t^2).
$$
Now write $|\psi(t)\rangle~=~e^{-i\omega t}|\psi(t_0)\rangle$, and use de Broglie-like relation $\omega~=~E/\hbar$, so that the energy $E$ is an eigenvalue of the Hamiltonian ${\hat H}|E\rangle~=~E|E\rangle$. We can see that the Hamiltonian operator is the Hermitian generator of the unitary time development operator $U(t)~=~e^{i{\hat H}t/\hbar}$. The Hamiltonian is then the generator that tells how a state evolves as $t~\rightarrow~t' > t$, and ${\hat H}~=~i\hbar\partial/\partial t$.
Suppose that time is an operator. We can now examine the energy development of a state $|\psi(E)\rangle~\rightarrow~|\psi(E + \delta E)\rangle$ and in the same manner we can see that the time operator is ${\hat T}~=~-i\hbar\partial/\partial E$. So far things seem alright. We can compute the commutator of the two operators acting on $|\psi(t)\rangle$ and $|\psi(E)\rangle$
$$
[{\hat T}, {\hat E}]|\psi(t)\rangle = [{\hat T}, i\hbar\frac{\partial}{\partial t}]|\psi(t)\rangle
$$
$$
= -i\hbar\left({\hat T}\frac{\partial}{\partial t}|\psi(t)\rangle~-~\frac{\partial}{\partial t}\big({\hat T}|\psi(t)\rangle\big)\right)~=~i\hbar|\psi(t)\rangle.
$$
Much the same works if we consider $|\psi(E)\rangle$ with ${\hat T}~=~-i\hbar\partial/\partial E$.
Consider a Hermitian time operator ${\hat T}$ such that $[{\hat T},~H]~=~i\hbar$, so a unitary operator $U_\epsilon~=~exp(-i\epsilon{\hat T})$ exists. This is an energy development operator, where $\epsilon$ is in the set of reals. The state $\psi$ in the eigenbasis of a Hamiltonian $H\psi~=~E\psi$, with commutator
$$
[U_\epsilon,~H]~=~\sum_{n=0}^\infty{{(-i\epsilon)^n}\over{n!}}[{\hat T}^n,~H]~=~-\epsilon U_\epsilon.
$$
defines the composite operator $HU_\epsilon$
$$
HU_\epsilon\psi~=~(U_\epsilon H~-~[U_\epsilon,~H])\psi~=~(E~+~\epsilon)U_\epsilon\psi.
$$
$U_\epsilon\psi$ is an eigenstate of the Hamiltonian with eigenvalue $E~+~\epsilon$. The Hamiltonian $HU_\epsilon$ is not discrete or bounded below, since $\epsilon$ has a continuum of values on the reals. If the Hamiltonian $H$ is discrete and bounded below the $U_\epsilon$ maps these eigenvalues on the entire set of reals, and the time operator does not exist
Define the time operator
$$
{\hat T}~=~i\hbar\sum_{j\ne k}{{|E_j\rangle\langle E_k|}\over{E_j~-~ E_k }}.
$$
that acts upon a ket $|t\rangle~=~N^{1/2}\sum_nexp(iE_nt/\hbar)$ as
$$
{\hat T}|t\rangle~=~i\hbar\sum_{j\ne k}{{|E_j\rangle\langle E_k|}\over{E_j~-~E_k }}|t\rangle~=~iN^{-1/2}\hbar\sum_{j\ne k}(E_j - E_k)^{-1}|E_j\rangle e^{-iE_kt/ħ}.
$$
This is a Fourier summation, which in the continuum limit gives $t|t\rangle$ by the Cauchy integral formula. Now compute matrix elements of $[T,~H]$ for $|\psi\rangle~=~\sum_ja_j|E_j\rangle$
$$
\sum_ja_j\langle E_i|[T,~H]|E_j\rangle~=~i\hbar\sum_{j,k,l}a_j\langle E_i|(E_k~-~E_l)^{-1}|E_k\rangle\langle E_l |E_j\rangle
$$
$$
=~i\hbar\sum_{j,k}a_j\langle E_i| (E_k~-~E_j)^{-1}|E_k\rangle,
$$
where the matrix element $\langle E_i|(E_k~-~E_j)^{-1}|E_k\rangle~=~\delta_{ik}(E_k~-~E_j)^{-1}$. $|E_i\rangle$ is not in the projective summation of the time operator for $[{\hat T},~H]~=~i\hbar$, and generally $[{\hat T},~ H]~=~0$.
A Cauchy sequence of states will converge to a bounded state $|\psi\rangle~=$ $\sum_{j=0}^Na_j(N)|E_j\rangle$, for $N$ the bound on the complete set. For the coefficient $\sim~(1/j)$ as $N~\rightarrow~\infty$ the accumulation point contains a dense set of points with $E~+~\delta E$ energy eigenvalues which satisfy $\sum_j\langle E_j|\psi\rangle~=~0$. This means that the commutator $[T,~H]~=~i\hbar$ holds on a measure zero set, and for the function $\psi(t)$ an almost periodic function.
