Seeing What is in Front of You (Appendix to my 2018 book, Philosophy of Time & Perceptual Experience)

Note: This post is a follow-up to the chapters in my 2018 book about visual experiences of spatially distant things, things that are at some depth in one’s visual experience.  

It is common to talk about the direction of vision, visual direction, or line of sight, and in general where things are located visually. It seems right to say that things seem to be located in a particular visual way to the seeing subject. In addition, there is a particular direction to this appearance, in that some things can be said to be visually off to the side or straight in front of the seeing subject. However, I have problems with deciding what actual or real direction should be used to evaluate this visual appearance. A consequence of these problems is I can only find bad reasons to hold that a visual appearance of direction is inaccurate.

This is a discussion about vision and space, not vision and time. But I think that what I’m going to say here relates to both. The issues for vision and space are very similar to issues for vision and time. The difference is that, for many theorists, time has a direction in a way that space does not.

Accuracy-Conditions of the Visual Appearance of Spatial Direction 

When I stand outside on a clear night with a full moon, the full moon will seem to be a certain distance and in a certain direction from me. This is a particularly visual experience: I (at least seem to) see the moon; I don’t (at least seem to) hear it, smell it, or touch it. One might then ask: is the appearance of the moon’s direction and distance accurate? I don’t like my answer but I don’t know what to do about it. My answer is: except in one possible special way, it is never inaccurate; it is always accurate.

The possible special way is this: if the appearance of the moon’s direction and distance is of it as absolute, then it is not accurate. I assume that apparent direction and distance is highly unlikely to correspond to an actual spatially absolute direction and distance. I do not think that there is absolute direction and distance. Furthermore, if there is an absolute direction and distance, then it would be a matter of chance that appearances correspond to it.

However, I also think that this isn’t clearly how we see the direction and distance of things. If we do, then we can say that this is an erroneous experience of the absoluteness of these spatial properties. For now, whether we have it or not, I’ll put that particular possible experience aside.

This leaves the question I’m interested in: under what conditions is the visual appearance of distance and direction accurate? I am interested in this question because, so far as I can tell, the answer seems to be one of the following:

(a) Visual appearance of direction is inaccurate when it fails to correspond to a relative direction in space.

(b) Visual appearance of direction is neither accurate or inaccurate because there is no fact of the matter about direction. ‘Direction’ is arbitrary.

(c) Visual appearance of (spatial) direction is always accurate because the definition or grounding of direction comes from the appearances, and nowhere else.

For those who want to say that the direction in which something seems is mistaken, (a)  seems best. As there is no absolute direction in space, any attempt to define direction is relative. As with distance or speed, no spatial entity is privileged with respect to spatial direction. It is relative to some points that anything is ‘far away’ or ‘moving fast’. It is also relative to some points that anything is ‘in front of’, ‘behind’, ‘above’, ‘below’, ‘to the left of’, and so on.

So, the appearances can be inaccurate with respect to relative direction. Yet, which relative direction? Determining that is problematic.

In general, appearances can be used to measure something. Appearances can play an epistemological role, where a theory of the world is evaluated by how closely it corresponds to appearances. In that case, appearance is somewhat like a measuring device, such as a metre stick. If we treat appearances that way here, then (c) is to be preferred: one evaluates the direction of something with respect to visual appearances.

However, here, we are asking about the appearances – about the accuracy of the measuring device itself. Ideally, one does not pick the thing being judged to evaluate it. That makes the accuracy trivial, like measuring a metre stick with itself.

So, what do we do? First, maybe we must bite the bullet here and use appearances, i.e., (c). Whether we like it or not, there is no better way of judging actual visual direction. But many may think otherwise: the other options surely can do something here. Either it is (b), there is a relative visual direction independent of appearances or it is (a), there is some absolute visual direction independent of appearances.

Honestly, I think it’s a toss-up between (b) and (c). What weighs the odds in favour of one or the other is whether you want to treat appearances seriously. If you are happy to throw them out (e.g., you hold that appearances never define or ground anything), then go for (b). If you are happy to have them play an epistemological role, go for (c).

However, note that (b) doesn’t provide a direction to evaluate visual appearances against. Instead, it is the position that there is no fact of the matter about a direction. The visual appearance is mistaken insofar as it seems to favour one direction over another. If you say that something seems to be in front of you, or to the side, your mistake is to hold that this could ever be something non-arbitrary – any kind of fact – one way or the other.

So, I think it can be (b) or (c), and I prefer (c) but can say little to dissuade someone convinced by the arbitrariness in (b).

However, I think my suggestion that only (b) or (c) are available should look obviously wrong to at least some readers. One should take (a) — judgement by relative direction — seriously.

For example, surely the very definition of mirages requires accuracy with respect to a relative direction.


Image on by Brahan Milla on Unsplash.

Mirages, as commonly described, are visual experiences where there seems to be something at a particular location in space which is not at that particular location in space. In the most obvious examples, the variation of appearance and reality is typically described as being visually inverted: that what visually seems to be facing one way is actually facing another.

So, in an otherwise empty desert, I see in a small region of the sky, hanging above the horizon in front of me, a street scene — one inverted so that the street occupants’ heads lie below their feet. This is clearly not how they are: they are not walking upside down in the sky. This is an inaccurate visual appearance.

That is a fairly straightforward way of describing the mirage. I imagine it would be risible for many to suggest otherwise. I’m going to suggest otherwise.

The reason I’m going to do that is the account of mirages is more complicated than that it ‘appears upside down and is not upside down’. Once the complications are included, I’m not sure that it’s right to say that things are not how they seem visually. I think that it’s better to say: things are how they visually seem but they do not match expectations of how I might interact with them beyond seeing them. Yet these latter expectations are based on my ignorance of how what I see and what I can touch may interact. This is a multi-sensory error, or an error of multisensory integration, but not of my visual experience itself.

Some examples of mirages:

Visual Illusion/Hallucination

(For more on this section, see my 2018.)

If I believe in elves that can be seen, then I do not hold that the visual appearance of elves is necessarily either an illusion or an hallucination. If I don’t believe in the moon, then I hold that the visual appearance of the moon is necessarily either an illusion (the moon is really something else with wrong apparent properties) or an hallucination (there is nothing there). What you believe is real partly determines your beliefs that how things seem is how things are, i.e., is accurate.

When some experience is an illusion or hallucination, it is in error in some way. As I’ve argued in an earlier post, how it is in error depends (at least in part) on how things appear (a phenomenological condition) and how things are really (a metaphysical condition). What one holds about the latter condition (at least) is theory-dependent. Thus, that there is an error — and what kind — is also theory-dependent.

The error can be different according to different theories even if: (a) everyone agrees as to what is really going on; they need only disagree about what appears to be going on; (b) everyone agrees as to what seems to be going on; they can disagree about what is really going on.

From this, I think that what is erroneous can depend on what one holds to be real about the senses themselves — and the properties we ascribe to them.

Turning this to errors in the direction of gaze (or visual direction), I think that there are three factors to consider:

(a) The definition of the direction of gaze

(b) The possibility of experiential error of direction which it neither illusion nor hallucination (anosognosia).

(c) The possibility of an experiential error of direction at all

Here is my reasoning for taking (c) seriously.

First, as argued elsewhere on the site, I don’t presume the universality of illusory counterparts. Just because something can be apparent doesn’t mean that it can actually be illusory. In the widest set of possible worlds, yes – it’s possible that what’s apparent is merely illusory — but it is an open question with respect to the actual world.

I think that the greater share of claims to error of direction are discrepancies between

(i) The apparent direction of something and

(ii) Its direction relative to a Earth-derived geometric system idealised as a sphere.

However, I also think that there is no particular reason to evaluate the accuracy of (i) by (ii). If one is to evaluate it by anything, it should be

(iii) A direction relative the path along which light travels from the source to the eye.

However, (i) is always accurate when evaluated with respect to (iii) light. That is, if something appears to be straight in front of you in mirages, loomings, or other strange visual phenomena, this is not, strictly speaking, erroneous. By the standard of the path of light itself, it is straight in front of you. This is the case if you choose the path of light from the source to your eye, the very thing that allows you to see the source. If you choose that path, then the source is straight in front of you and even orientated as it appears to be; that apparent direction and orientation is not mistaken.

(You might choose some other path of light to evaluate it, and judge it wrong. Fine — but why? One path of light is as good as another. If you are arbitrarily doing so, then the error here is at best in the absoluteness of apparent direction and orientation (and distance — but I’m ignoring that in this post)).

I think that will sound very wrong to many readers. However, I think, when it comes to considering the alternatives — the body, the world, the head etc. etc. — light is still the better option. All other options lead to cases of error that are intuitively not error. This option leads to cases of veridicality that are intuitively error. Since these cases are also cases of perceptually apparent veridicality (simply by being perceptually apparent) then we have a clash between (a) appearances of veridicality and (b) intuitions of error. In such a clash, where neither side is incoherent, I pick (a) every time.

Let’s look at some alternative ways of picking out the correct direction to judge experienced direction.

Where The Eyes Are

My eyes lie a certain distance apart from each other. They can face different directions.

My eyes are not exactly in the same spatial location.  Not only are they not in the same spatial location, they do not always face in the same direction. One is much weaker and slightly squinted; when I am tired, it points a little inwards toward my nose. So,

  1. Each of my eyes is In a different place.
  2. At least sometimes, each of my eyes points in different directions. [Note 1]

‘1’ assumes eyes occupy a location in this space (this seems uncontroversial, and I’ll assume it). Further, that each can occupy a different location — again, uncontroversial and I’ll assume it. ‘2’ holds that my eye is pointing in a particular direction. But this is not as clear as ‘1’. It looks like a right thing to say. But: what gives an eye that direction? There are three possible answers:

A. An eye has an intrinsic direction.

It has a direction whatever else we define as having direction. If an eye is floating in an otherwise empty universe, it still points or faces in some way such that we can say: there is one whole heap of nothing in front of the eye and there is another whole heap of nothing behind the eye.

An eye’s direction of this kind is like the direction of an arrow or a signpost. There is a part of it that lies forward of the eye — analogous to the head of an arrow — and a part that lies back in the way — like the tail of an arrow. [Note 2]

B. An eye has an extrinsic direction.

Its direction is because of something else. It needs something else to say where it is pointing. If an eye is floating in an otherwise empty universe, it can’t be said to have anything lying in front or behind of it.

It is not like an arrow. It is like a line. We say what is at the front because of something else — something else which is like an arrow — and that something else gives it its direction. Draw an arrow and a straight line segment, and put the line segment in relation to the arrow in such a way that part of it is nearer the arrowtail than the ‘head, and part of it is nearer the arrowhead than the ‘tail. If asked ‘where is the segment’s front and back’, my bet is you’ll find it natural to have the front part near the arrowhead, the back part near the arrowtail. [Draw it. You’ll see what I mean].

C. Eyes have no direction, neither intrinsic nor extrinsic*

*I am not sure what to do with that.

If ‘A’ and ‘B’ are right, then we can then also define spatial directions relative to the eyes. If an eye has a direction, we can talk about what lies along that direction, what lies to the left and right of that direction, what lies above and below that direction.

We will move on to the idea that ‘A’ is correct next, and how it applies to defining the visual direction.

But if ‘B’ is correct, then something else defines the eye’s direction. So, what defines that direction?

Defining ‘In Front of the Eye’

What makes one thing, e.g., an apple I’m looking at, in front of the eye while making something else, e.g., the neural processing resulting from stimulation of the eye, behind the eye?

Here are some options: the earth (or some bigger thing, even); the body; the head; the eye-brain system; a specific set of neurons; the ‘self’; a straight line running through the eye. Let’s take each in turn.

The Earth/The Earth’s Orbit/The Solar System/The Heliosphere/The Galaxy/The -[insert large embedding spatial entity here]

I can’t make sense of the idea that ‘in front’ for my eye is relative to the earth or any other larger body — to which, for all intents and purposes, we are at rest and in which we are located or embedded. Instead of going through all cases (orbit, heliosphere, etc.), let’s just take the Earth.

‘Directions’ on earth are typically defined in terms of latitude/longitude and North/South/East/West. Let’s ignore for the sake of argument the conventionality of these terms; for example, if we didn’t ignore it, I think we could as much define directions by the rolling of the Earth on its axis about the sun: say, descent-wise (in the direction in which the Earth turns toward the sun) and ascent-wise (the direction in which the Earth turns away from the sun.

To say that directions on the Earth define ‘in front for my eyes’ is to say that something in front of my eye because it is either North/South, etc., of my eye. This means that, no matter how I turn my eyes or head, or body, something remains in front of my eye so long as it is the same relative to the directions defined by the Earth.

I take it that this is just flat-out wrong. But I also think that most other directions, though commonly closer matches to the eye’s direction, e.g., the body, head, etc., are equally as problematic. They have the same general problem: what we call ‘in front of’ for the eye can vary from whatever we might call any direction as defined by them.

The Body

Say I define ‘in front of my body’ as the direction shared by (a) the direction in which I walk and (b) my knees move (walking ‘backward’, my knees move in the opposite direction to which I walk). I can turn my head away from that direction. Now, what is in front of my body is then not in front of my eyes. Also, this definition of the body’s direction seems quite forced to me (but I’ll put that aside for now).

The Head

Say I define ‘in front of my head’ as the side closest to the majority of sense-organs (eyes, tongue, nose, ears (just)). I can twist my eyes to the left without turning my head; what is in front of my head is not then in front of my eyes.

The Eye-Brain(or Eye-[Specific set of Neurons]) 

One idea might be this: the eye and something to do with the brain (the whole brain, or a particular set of neurons/neural processes) define the direction on which to judge something as being ‘in front’ and ‘behind’. To the question above: ‘What defines neural processes as being  behind the eye?’, the answer is trivial: the direction is defined by the processes being behind the eye. As it were, they are the tail of the direction-defining arrow. In that answer, we might treat it as an unanalysable given that they so lie behind. There is a kind of intrinsic, basic, fundamental direction that the eye-brain system has, such that what is in front of and behind can be derived from it. And then from that, we can define what is in front of the eye, or even that the eye is at the front of the system.

So what picks out the direction? How about this: the eye+brain system has a line from the back to the front of it which can be defined as straight. But what is the ‘front-back’ here? The eyes are in front, the rest of the brain behind it.

Here is a gross counter-example I hope you never have to participate in: I can technically move one eye — say, my left — such that it faces into the connected brain (some already have had it happen as far as looking down the cheek). Now stretch the left eye so that it not only faces into the brain but passes through or over it — so that its facing out from the brain again in the opposite opposite to the right eye. If all of this was happening in the head, I would now have my left eye looking out through the hair at the back of my head.

Now, I have one eye facing backwards according to the other eye. What is ‘in front’ of each eye? An eye-brain system gives two different answers, one for each eye.

However, it does not do so independent of an understanding of the answer independent of thoughts about the brain. It takes it from their already determined direction. And that’s good — because I can’t see any good reason to suppose that either the left or right dominates here as determining eye-brain direction. Here, I’d say the eye-brain system faces in two different directions, with very different things ‘in front’ of it. And that’s because of the directions the eyes face, and what’s in front of each of them. (If still thinking about this, think also about pigeon’s eyes — and ask what’s in front of each one of them). What is said here for the brain goes also for any part of the brain, including the any specific set of neurons.

The ‘self’

Similar to the eye-brain, we might say that a self-eye system defines ‘front’ for the eye. What is behind the eye is whatever lies on the same side of the eye as the self; what is in front of the eye is whatever lies on the other side.

However, where is the self so that you can even say what lies some distance or direction from it? I have no idea.

One general idea of physically-minded theorists is that, if it exists at all, the ‘self’ is in the brain. So what we’re saying is: a part of the brain + eye. See above for problems with that.

All appeals to the brain have a general problem, one brought up by Dennett in his early work.

Looking at Your Brain In Its Jar

In Dennett’s Brainstorms, Dennett tells a story about a situation where his body and brain are remotely linked, such that his brain is kept in a room while his body roams around out in the world; his sense-organs go with his body. In this story, where Dennett — himself — is seems to be where his body is. Yet, it is if anywhere, where his brain is. In the story, he is shown his brain. He stands there looking at the brain — him — in the jar. Is he standing in front of himself? If so, then, what is in front of his eyes?

If we take the self to be in the brain, and ‘in front’ for the eyes to be defined by whatever lies on the other side of the self, we get this: Dennett’s skull, and what Dennett does not see, is in front of his eyes. While what he sees — e.g., the room, the walls of the jar before his body — this is behind his eyes. If he says ‘what lies before my eyes is my brain in a jar’ he is strictly speaking wrong. This lies behind his eyes as is a lot of what he sees.

All of these solutions share a similar feature: they involve evoking something other than the eye alone. The eye inherits its direction from either something else — the earth, the body — or from something else along with the eye. Perhaps this is too complicated. We can take the eye on its own. We can say that the eye has an intrinsic direction.

A Straight Line Running Through the Eye 

This looks like the ideal candidate. The eye has structure. It has a lens, retinal cells, and they all lie on different parts of the eye. We define direction this way: the retinal cells are further back than the lens; the lens are further forward than the cells.

Further, lens and cells make a line so that we can say what lies off that line is to the left/right/above/below of the line. If we define direction relative to this structure, it seems to survive every counter-example so far. If I turn my head, my eyes, bend one out the back of my head, relocate my self, still: retinal cells lie further back than the lens, and the lens lies at the front.

However, there are still problems, indicated by the following questions:

(1) There are many lines in the eye; the eye is an ovoid; none obviously stand out. Why pick this line running through lens/cells? Why not pick ‘front’ as what lies orthogonal to that line — to the side of the line?

(2) It’s a straight line. But there are many straight lines passing through the eye.

(3) What makes that line straight rather than crooked or curved?

Isn’t the Answer Obvious?

Finally, one might simply say: it’s obvious what defines the right direction for judging visually apparent direction. So, putting aside everything I’ve said so far (don’t appeal to body, world, head, etc.), what is that is obvious? I don’t know.

The ‘veridical’ in ‘how things visually appear in veridical perception’ depends in part on veridical direction.  Things visually seem to lie in front of us. So, what determines the veridicality of that ‘in front’?

Pick any location in space — any location at all

The location can be completely arbitrary; you are free to put it anywhere. For the sake of this discussion, call this location ‘ansin‘. Ansin can have two spatial properties relative to each of my eyes:

(a) The spatial distance between each eye and ansin.

(b) The direction of ansin relative to each eye.

How do we determine ‘b’ — the direction of ansin relative to each eye? Is Ansin the same distance and direction from each eye?

First,  yes, ansin is trivially the same distance and direction from each eye — if each eye is in the same place as the other eye. However, each eye is not in the same place as the other.

Second, yes, ansin is at least practically the same distance and direction — if the distance between ansin and each eye is so far that the differences in their locations serve no practical purpose.

For example, say ansin is the Andromeda Galaxy, or some point within it. Say my eyes are a few inches apart, both point the same way, and I am looking sideways at Andromeda. Strictly speaking, ansin lies at a different distance and direction to each eye, but it’s a tiny tiny difference.

It doesn’t work so well, though, if ansin’s distance is comparable in scale to the distance between the eyes. For example, if it’s a dust mote floating by my left eye.

Nor does it work if my eyes point in different directions. If I look cross-eyed at the sky, and one eye faces ansin in Andromeda directly, even if it looks to the same distance, my other eye is likely looking at a very very different point in space — one light years away from Andromeda. Differences in direction increase differences in position the further out you go.

You can probably tell a similar story about ears, and hands, and parts of the skin, bits of the nose, areas of the tongue, nerves in the body and whatever else you hold to be the set of organs related to (a) any particular sense and/or (b) all the senses together.

I think that this fact –that the organs of each sense (and all of them together) are distributed in space — matters. It matters when we come to saying that there is some error in the resulting perceptual experience. In particular, it matters when we come to classify any such error as illusion or hallucination or even as an error at all. I think the following is always a possibility: we interpret the experience as mistaken because we do not consider some limit to the experience itself. Our decision is influenced by how we understand the experience to be related to its sense-organs, and how we understand these organs to be distributed in space.[Note 3]


1. Direction and magnitude — the engineering/mathematically savvy will recognise vectors here. I don’t want to explain vectors here, and prefer to discuss the direction/distance issue separately.

2. A direction defined by what? This question is especially relevant given that I am talking about the direction of other things such as ansin, arguing that ansin has a different direction relative to each eye. So what determines that each eye has a different direction?

I think anything will do, even each eye, with the condition that, if the eyes have different directions relative to at least one other thing, then they have different directions; they may lie in the same direction, however, relative to other things, and depending on different ways of calculating coordinates in space, e.g., lying in different directions under straight Newtonian coordinates, but parallel under curved space.

It’s like with motion: if two bodies have different speeds relative to something, This makes direction relative, conventional and/or arbitrary, depending on how one picks out or chooses to work with ‘anything else’.

However, one might want a general answer. So, perhaps ‘anything’ could for example refer to everything — and one could say that the direction of the eye is relative to what it is relative to everything. To make that coherent, because many things face in different directions relative to each other, either this ‘relative to everything’ would have to be:

(a) A whole range of values — making the eyes ‘direction’ a set of varying directions.

If the range of values is all possible values, then that set includes all directions. So, in which direction is my left eye facing? Answer: any direction whatsoever (but depending on what one is determining it from).

(b) A calculation (an average, mean, etc.) from these values.

(c) Relative to everything within some domain of things, e.g., electrons in the Earth’s magnetic field.

One could try this: face the direction in which they move (if one wants to anthropomorphise them, and not only that, but to a normal human — no running backwards). Then, a direction ‘relative to everything’ is restrictively defined as being relative to a coordinate system determined by the average of those electron’s velocities (getting Magnetic North). useful for certain contexts.

But this is not the direction which one is forced to accept in all contexts. If I prefer to define direction according to something other than this average, I don’t see why I should also consider this direction as well.

For example, say I want to define direction relative to the path light takes between Betelgeuse and my eye when that light hit my eye one night late last autumn.  I say what’s left or right, above or below, in front and behind. That looks just fine. To avoid questions of relativity, one might argue for a non-relative, or non-arbitrary and non-conventional direction to which one can then define the direction of each. But, unlike with time,  I don’t know anyone who thinks there is such a direction in space (do you?).

3. I refer to the set of conventionally-identified organs related to one sense as just one organ for that sense. E.g., the sense organ for sight is the set of eyes; it is not each eye (which we conventionally identify as a single organ). Why?

I think the common reasons for picking out each organ is relative to the picker’s discriminatory limits. They identify two sense organs for sight because the visual system on the surface of the body, on the human interactive scale, seems to have two parts — the eyes. But how it seems to someone is no reason on its own to think that the visual system is comprised of one, two or an multiple of sense organs.

It may seem that way because of limits of the perceiver only if how it seems to someone is how it is are we so justified (and then trivially, since in such a reason we build in ‘how it is’ to justify ‘how it is’). Putting aside how they seem to others, it’s ambiguous if each is:

(a) An entire sense organ of its own. For example, because each eye is somewhat free to move on its own. If sense organs are identified as capable of independent movement, then maybe you can say that each is a separate organ.

(b) A set of sense organs or part of a larger sense-organ. Call each eye a very large set of sensory organs — a light detectors.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.