Problem: Given Newton's second law
$$ m\ddot{q}^j~=~-\beta\dot{q}^j-\frac{\partial V(q,t)}{\partial q^j}, \qquad j~\in~\{1,\ldots, n\}, \tag{1} $$
for a non-relativistic point particle in $n$ dimensions, subjected to a friction force, and also subjected to various forces that have a total potential $V(q,t)$, which may depend explicitly on time.
I) Conventional approach: There is a non-variational formulation of Lagrange equations
$$ \frac{d}{dt} \left(\frac{\partial L}{\partial \dot{q}^j}\right)-\frac{\partial L}{\partial q^j}~=~Q_j, \qquad j~\in~\{1,\ldots, n\},\tag{2} $$
where $Q_j$ are the generalized forces that do not have generalized potentials. In our case (1), the Lagrangian in eq. (2) is $L=T-V$, with $T=\frac{1}{2}m\dot{q}^2$; and the force
$$ Q_j~=~-\beta\dot{q}^j\tag{3} $$
is the friction force. It is shown in e.g. this Phys.SE post that the friction force (3) does not have a potential. As OP mentions, one may introduce the Rayleigh dissipative function, but this is not a genuine potential.
Conventionally, we additionally demand that the Lagrangian is of the form $L=T-U$, where $T=\frac{1}{2}m\dot{q}^2$ is related to the LHS of EOMs (1) (i.e. the kinematic side), while the potential $U$ is related to the RHS of EOMs (1) (i.e. the dynamical side).
With these additional requirements, the EOM (1) does not have a variational formulation of Lagrange equations
$$ \frac{d}{dt} \left(\frac{\partial L}{\partial \dot{q}^j}\right)-\frac{\partial L}{\partial q^j}~=~0,\qquad j~\in~\{1,\ldots, n\},\tag{4} $$
i.e. Euler-Lagrange equations. The Legendre transformation to the Hamiltonian formulation is traditionally only defined for a variational formulation (4). So there is no conventional Hamiltonian formulation of the EOM (1).
II) Unconventional approaches:
Trick with exponential factor$^1$: Define for later convenience the function $$ e(t)~:=~\exp(\frac{\beta t}{m}). \tag{5}$$ A possible variational formulation (4) of Lagrange equations is then given by the Lagrangian $$ L(q,\dot{q},t)~:=~e(t)L_0(q,\dot{q},t), \qquad L_0(q,\dot{q},t)~:=~\frac{m}{2}\dot{q}^2-V(q,t).\tag{6}$$ The corresponding Hamiltonian is $$ H(q,p,t)~:=~\frac{p^2}{2me(t)}+e(t)V(q,t).\tag{7}$$ One caveat is that the Hamiltonian (7) does not represent the traditional notion of total energy. Another caveat is that this unconventional approach cannot be generalized to the case where two coupled sectors of the theory require different factors (5), e.g. where each coordinate $q^j$ has individual friction-over-mass-ratios $\frac{\beta_j}{m_j}$, $j\in\{1, \ldots, n\}$. For this unconventional approach to work, it is crucial that the factor (5) is an overall common multiplicative factor for the Lagrangian (6). This is an unnatural requirement from a physics perspective.
Imposing EOMs via Lagrange multipliers $\lambda^j$: A variational principle for the EOMs (1) is $$L ~=~ m\sum_{j=1}^n\dot{q}^j\dot{\lambda}^j-\sum_{j=1}^n\left(\beta\dot{q}^j+\frac{\partial V(q,t)}{\partial q^j}\right)\lambda^j.\tag{8}$$ (Here we have for convenience "integrated the kinetic term by parts" to avoid higher time derivatives.)
Doubling trick: See e.g. eq. (20) in C.R. Galley, arXiv:1210.2745. The doubled Lagrangian is $$ \widetilde{L}(q_{\pm},\dot{q}_{\pm},t)~=~\left. L(q_1,\dot{q}_1,t)\right|_{q_1=q_+ + q_-/2} -\left. L(q_2,\dot{q}_2,t)\right|_{q_2=q_+ - q_-/2} + Q_j(q_+,\dot{q}_+,t)q^j_-\tag{9}. $$ The initial conditions are $$ q^j_+(t_i)~=~q^j_i,\qquad\dot{q}^j_+(t_i)~=~\dot{q}^j_i\qquad\text{and}\qquad q^j_-(t_i)~=~0.\tag{10} $$ The final conditions are $$ q^j_-(t_f)~=~0\qquad\text{and}\qquad \dot{q}^j_-(t_f)~=~0 \qquad\Rightarrow\qquad\left.\frac{\partial \widetilde{L}}{\partial \dot{q}^j_+}\right|_{t=t_f}~=~0 .\tag{11} $$ The $5n$ boundary conditions (10) & (11) do not overconstrain the system. One still gets the Lagrange equations (2) [now posed as an initial value problem!], and the physical limit solution $q_-^j= 0$. The doubling trick (9) is effectively the same as introducing Lagrange multipliers (8).
Gurtin-Tonti bi-local method: See e.g. this Phys.SE post.
--
$^1$ Hat tip: Valter Moretti.