Simplest possible thought experiment that illustrates the difference between the past and the future? We are led to believe that the "arrow of time" is determined by the direction in which entropy increases. Entropy is a measure of disorder, but disorder is very much in the eye of the beholder.  Is there not some experiment that reduces to its most basic elements the reason that we can remember the past but not the future.  Why is it that events in the past seem to leave a clearer imprint on the present than events in the future? Why is it trivial to store a bit of information about a past event but not about a future event? Maybe there's something fundamental about the behavior of a bistable memory latch?  I'm looking for the very simplest possible thought experiment that captures the essense of this phenomenon.
[Edit] This topic and the entropy arguments keep bothering me so I'll add a bonus in the hope of getting more attention. I see there's also a Wikipedia article  and this includes a reference to a toy model about fleas and cats but again it reduces to a low entropy initial condition. There is also a statement that the past is what we can remember and the future is what we can influence. Is this some kind of perverse symmetry that we fail to recognise? - but I'm looking for physics not philosophy! 
Brownian motion is about as simple as it gets.  When I measure the position of a particle, it tells me something about its position, say, 1 second earlier. But it seems to tell the the same amount of information about where it will be 1 second later. So it's a simple example of a irreversible dissipative system that has memory - but it's completely time-symmetric.  
Another observation concerns Markov chains.  If I make the process go backwards instead of forwards, the labels change.  But this seems to be because we label the arrows with conditional probabilities. If we were to label the arrows with absolute probabilities the labels would be unchanged when we reversed time. So this is just a curiosity.


[Edit #2] Maybe the most trivial example is thermal inertia. The temperature of a large thermal mass will lag the temperature of its environment.  The current temperature of the mass has information about the temperature of the environment in the recent past, but it says nothing about the future environment. I suppose this simply says heat flows from hot to cold - which comes back to entropy.

[Edit #3] The idea that thermal inertia provides a fundamental example of the asymmetry of time seems to crumble once it's examined closely.  The temperature of a thermal mass (in a gas) strictly does not give information about the recent temperature of the gas, it gives information about the average kinetic energy of the atoms that recently hit the mass. It gives exactly the same amount of information about the kinetic energy of the atoms bouncing off the object in the near future.   
[Edit #4] Maxwell's demon seems to have some relevance here. The demon can switch on or off a large energy barrier separating the two compartments. The usual focus is on the information needed to switch the barrier at the correct instants.  But the sudden change in the energy landscape is of interest in its own right. A trivial situation would be a single atom bouncing randomly between two states when a barrier is suddenly introduced. This captures one bit of information and stores it as long as the barrier is maintained. This seems like the essence of memory. The introduction of the barrier is time asymmetric. But, the removal of the barrier is the exact equivalent in backwards time. Also, the use of the words introduction and removal are problematic here because they define the direction of the time asymmetry. 
[Edit #5]  particle in box with sliding partition 
This picture shows the sort of answer I am after.  This is an open irreversible system. Here we have a single particle in a box and everything is in thermal equilibrium.  There is a partition that is positioned to be either outside the box or to be inside the box (dividing the box into two separate volumes). The partition changes position once.  We can consider the time axis to be divided into two portions. In one portion of time, the particle can move between the two sides of the box. In the other portion of time, the particle is always in either the left or right side of the box. Note that, so far, we have been careful to say nothing to imply which way time is flowing. 
In one interpretation, the partition is intially open (time flows from left to right). So the system can be viewed as forming a permanent memory of where the particle was around the time the position of the partition changes. 
In the other interpretation, the partition is intially closed (time flows from right to left).  So the system can be viewed as having permanent intention to influence where the particle will be around the time the position of the partition changes. 
There are still the usual Maxwell demon problems here too. The sliding partition can be arranged to not require energy to move between the two positions, but to reliably open (or close) the latch that releases (or captures) the partition will require an energy source significantly larger than thermal energy.  But the problem of figuring which half of the partitioned box the particle is in seems to go away. The particle isn't going anywhere, so we can make multiple noisy measurements until we're convinced of which half it's in. 
I realize this doesn't answer the question, but it perhaps provides some food for thought.
 A: I suggest a pragmatic interpretation of the question: Design an experiment E such that a discriminator D (such as an individual or team of scientists) can determine whether a recording R of E is being played forward or backward. The harder it is to fool D, the "better" E is, for our purposes. Perhaps this could be phrased more precisely in information-theoretic terms.
There are many examples that can be used for E. A classic example comes from thermodynamics: Put two bodies in contact, and heat spontaneously flows from the warmer object to the cooler object, not viceversa. If you observe heat flowing to the warmer object, you, as a discriminator, can be pretty sure that you're seeing a backward recording.
An even more extreme example of this is opening a container full of colored gas. If you see the gas expand and dissipate, you can be pretty sure the recording is being played forward. If you see it do the opposite, shrinking in volume and into the container, you can be pretty sure the recoring is being played backward. It would be extremely difficult—something akin to a "great conspiracy"—to set up initial conditions such that the gas would do such a thing, under ordinary time evolution. See the SEP article on Thermodynamic Asymmetry in Time for more on the subject.
There are electromagnetic and gravitational analogues to the above in terms of incoming/outgoing electromagnetic or gravitational radiation, where the latter escapes to infinity. Connections can be made along these lines to some astrophysical/cosmological concepts, such as the Vaidya metric and white holes. Tangentially related: This answer to the question "How do you prove you're from the future?".
There is also a possible connection between quantum entanglement and the arrow of time, the idea being that the former (increasing entanglement) may explain the latter.

Using an obscure approach to quantum mechanics that treated units of information as its basic building blocks, Lloyd spent several years studying the evolution of particles in terms of shuffling 1s and 0s. He found that as the particles became increasingly entangled with one another, the information that originally described them (a “1” for clockwise spin and a “0” for counterclockwise, for example) would shift to describe the system of entangled particles as a whole. It was as though the particles gradually lost their individual autonomy and became pawns of the collective state. Eventually, the correlations contained all the information, and the individual particles contained none. At that point, Lloyd discovered, particles arrived at a state of equilibrium, and their states stopped changing, like coffee that has cooled to room temperature.


According to the scientists, our ability to remember the past but not the future, another historically confounding manifestation of time’s arrow, can also be understood as a buildup of correlations between interacting particles. When you read a message on a piece of paper, your brain becomes correlated with it through the photons that reach your eyes. Only from that moment on will you be capable of remembering what the message says. As Lloyd put it: “The present can be defined by the process of becoming correlated with our surroundings.”

Perhaps the kinds of simple systems mentioned above might satisfy your criteria.

See also Relation between the psychological and thermodynamic arrows of time (2014) by Leonard Mlodinow and Todd A. Brun:

In this paper we lay out an argument that generically the psychological arrow of time should align with the thermodynamic arrow of time where that arrow is well defined. This argument applies to any physical system that can act as a memory, in the sense of preserving a record of the state of some other system. This result follows from two principles: the robustness of the thermodynamic arrow of time to small perturbations in the state, and the principle that a memory should not have to be fine-tuned to match the state of the system being recorded. This argument applies even if the memory system itself is completely reversible and nondissipative. We make the argument with a paradigmatic system, and then formulate it more broadly for any system that can be considered a memory. We illustrate these principles for a few other example systems and compare our criteria to earlier treatments of this problem.

There are also articles about it here (section titled Memory Systems) and here.
A: No complicated mathematics or physics are required to fundamentally answer the question posed.
No thought experiment is required . The present gives information about the past , and the present, in a profoundly different way than that information speaks about the future. Although the future can be predicted by numerous methods,  the missing piece of the prediction makes the distinction between past and the future obvious. Events occurring in the far reaches of the universe,  of which we have no knowledge,  can intersect the predicted future in ways that would alter the future prediction and produce states that could not be otherwise incorporated in the prediction based on past or present conditions. Past events are capable of being known while future events can only be predicted. The difference between past and future is completely based on this information,  which can not be known until it has occurred in the present. The future is different from the past for the simple reason that,  to spite prediction,  we don't really know what will happen a moment from now.
A: I submit that the fundamental difference between the past and the future is not that the past is remembered, but that the past is forgotten.
On the macro scale, in the examples given by others involving "playing the movie backwards," what usually gives away the fact that it's played backwards is that the system exhibits more heterogeneous detail in the past.  Diffusion is the quintessential example:  the diffusion/heat equation indicates that in the forward direction, curvature (second derivatives with respect to space) gets smoothed out, that is, spatial fluctuations in temperature or density are decreased over time. If you reverse time, there is a minus sign introduced in the equation that says the fluctuations (spatial inhomogeneities) will grow.  But that equation governs macrostate variables that are already local averages, so the paradox is why such asymmetry exists for those macrovariables when the underlying microscopic dynamics are time-reversible.
That suggests a way to approach thought experiments on the micro scale along the lines I think you were asking about.  What you want is a scenario where a system begins in some state and time is allowed to move forward.  Then, after some time, the velocity of every particle is instantaneously reversed and the system continues to evolve.  If time is symmetrical then the system would return to its initial state.  The arrow of time would be indicated by any system that would not under velocity reversal (and reversal of anything else that depends linearly on time) return to its initial state.  Equivalently, there would be some way to tell that the velocities had been reversed rather than having run time backwards.  I'll give two examples:

*

*Two electrons approach each other, collide, and recede.  During such a collision, the electrons accelerate, and so emit radiation.  If after the collision you reversed the velocities, the collision itself would play out in reverse, but there would be additional radiation emitted, whereas if you played the movie backwards, there would be radiation coming in from infinity and being absorbed by the electrons.


*Consider a Maxwell's demon that allows gas particles to go from a left chamber to a right chamber, while particles incident on the gate from the right get reflected back into the right chamber. (Really, it doesn't have to be a demon, just a one-way valve will do, but making a one-way valve on the microscopic scale is itself nontrivial.)  Allow the demon to operate for a time, then reverse the velocities of all the particles, and the particles that had initially passed from the left chamber to the right will not be able to return to the left chamber, as they would if you played the movie backwards.  If you also require that the demon get reversed when the velocities get reversed, allowing particles from right to left instead of left to right, you still won't return to the original state, because particles from the right that were originally excluded from passing through will now be admitted.  You will not only restore the particles that originally passed from left to right back into the left chamber, but you will also allow some other particles from the right chamber into the left that were not there originally.
A: Can I remember an event in the future ? Yes, if I can travel backwards in time. Or even just make a timestamped record of the future event which I send backwards in time and then read/view in the past. Since there is nothing in physics that absolutely prohibits backwards time travel (although it seems to be very difficult and energetically expensive), there can be no simple thought experiment that proves you can only remember the past.
A: In a block universe, spacetime is analogous to a 2D-carpet with predictable patternwork. The pattern at the present location tells you perfectly the pattern in any direction around you (the past and future). The laws of physics are completely deterministic, so we can make this analogy. You can calculate, with probabilities, using the physical laws, what the pattern looks like in any direction, including the future direction. And you can recall patterns you have already seen. So “why is it trivial to store a bit of information about a past event but not about a future event” as you asked? Well, you interacted with, or you entangled with, a local patch of spacetime. And in that interaction, part of what occurred was a biophysical process of memories being formed in your brain. You don’t form memories of the future, or distant parts of the carpet you haven’t been to, because you haven’t interacted, entangled with, or been local to them yet. You can calculate via the equations of physics what future patterns will be, but that isn't "remembering" them. This highlights the importance of locality of interactions. Things interact when they are close in spacetime.
There is much more to say (why do we perceive a flow of time in a block universe, the arrow(s) of time/second law), but I wonder if any of that is needed to answer your original question to be frank
A: Our consciousness aligns with only  one of the two entropic arrows of time. Or at least that is what we think. Perhaps the process is perfectly symmetric and there is another consciousness that sees time forward in the opposite direction. They can make sense of the world in reverse in the same way we can make sense of the world in direct. Direct just meaning direction in which entropy increases. In their world, entropy always, or most of the times, decreases.
A: The simplest possible thought experiment is the one you have already given. I think I can remember the past, but I don't think I can remember the future.
But you are asking something different, for which physics has no explanation. The known laws of physics are time reversible. Even entropy is technically time reversible. Given a statistical state of low entropy at particular time and no outside interference, entropy would increase both forwards and backwards in time! We simply dismiss this as absurd, and assume that a state of low entropy can only arise as an initial condition created by outside interference. As far as we know, this is a very good assumption. However it has more to do with common sense than the fundamental laws of physics. After all, in any physics experiment we set the initial condition of a system using outside interference. You are really asking for an experiment which does not do that. Such an experiment is unknown to physics.
I did come up with one thought experiment, based on the model of an expanding-contracting universe. In such a universe the initial, Big Bang, state is identical to the final Big Crunch state. Both are identical singularities, so entropy must be identical in both states. In this case one would think that the law of entropy in the collapsing universe must be determined from the Big Crunch, in which case it would be reversed in the collapsing universe. Then a civilisation could in the collapsing universe would experience time backwards according to our idea of future and past. Such a civilisation could leave an artefact on a dead planet, and at some time in the future we could go to that planet and find it. Then we could destroy it, meaning that the future civilisation could never have left it.
As far as I can see, the existence of such a paradox, in a universe which obeys the known laws of physics (even if it is not our universe), shows only one thing. That we do not know the fundamental cause of the difference between past and future.
A: I personally feel like the answer is buried in Gibbs' derivation of the canonical ensemble.
You can start with something like this, at the microscopic scale, dealing with a multitude of hard Newtonian particles,
$$\int\int...\int\int P\ dp_1\ dx_1 ...\ dp_n\ dx_n=1$$
Which, upon realizing that the probability, P, is a function of all the positions (x) and momenta (p), we can define statistical equilibrium as the condition where,
$$\frac{dP}{dt}=\sum^{n}_{i=1}\Big(\frac{\partial P}{\partial x_i}\frac{\partial \epsilon}{\partial p_i}-\frac{\partial P}{\partial p_i}\frac{\partial \epsilon}{\partial x_i}\Big)=0$$
Note: I'm using $\epsilon$ for system energy because I use capital E for electric field a lot.
So at this point in the mathematics everything is completely time reversible:
$$dx_i =\dot{x_i}dt,\ \ \  dp_i=\dot{p}dt$$
Solving the above PDE gives the canonical ensemble,
$$P(\epsilon)=exp((F-\epsilon)/k_BT)$$
And from here all of the equations regarding entropy fall out. I would say that at this point the system is measured as an irreversible process. My intuition on it is that zooming out to the macro scale is where we lose time symmetry. Since only macro details of the system can be observed, it's no longer reversible.
A: If you accept that the waveform is not real and that it represents the observers knowledge of the system, then every QM measurement highlights the difference between the past and the future. Perhaps the real puzzle is why systems behave the way they do under the transformation t → -t.
A: 
Another observation concerns Markov chains. If I make the process go backwards instead of forwards, the labels change. But this seems to be because we label the arrows with conditional probabilities. If we were to label the arrows with absolute probabilities the labels would be unchanged when we reversed time. So this is just a curiosity.

There is no absolute probability, probabilities are always conditioned by what we know at the time. If we could know the future all future events would have a probability of 0 or 1, just as those in the past –there would be no Markov processes, everything would be deterministic. So you are actually making a profound point: probabilities would be nonsensical if the arrow of time were reversible.
If "time" is linked to "knowledge", then it is linked to "us," the observer. So let's take the athropocentric view that each time we learn something new, as when we make a new measurement or observation, the clock ticks. If we never unlearn, the counter always moves forward. If we are capable of unlearning by erasing the memory of a learned event, the counter would go back –but we would never know we traveled back in time. When we relearn it, the unlearned event will look just as new and exciting as it did the first time.
