Locally connected exceptional minimal sets of surface homeomorphisms

Andrzej Biś^{†}, Hiromichi Nakayama^{‡} and Paweł Walczak^{†∗}

Abstract

One of substantial problems in dynamical systems is to characterize the topological types of minimal sets. In this paper, we are concerned with locally connected minimal sets of surface homeomorphisms. In the case of a homeomorphism of a closed orientable surface different from the torus, we obtain that its locally connected minimal set is either a finite set or a finite disjoint union of simple closed curves. On the other hand, there is a homeomorphism of the torus with a locally connected minimal set which is neither a finite set nor a finite disjoint union of simple closed curves nor the whole torus. We will show that, if such a minimal set exists, then the minimal set is unique and it satisfies conditions similar to those of the Sierpiński curve.

In 1977, Fathi and Herman ([5]) proposed the following problem: Does there exist a C^{∞} diffeomorphism f of a compact manifold such that f admits a minimal set which is not locally homeomorphic to neither a Euclidean space nor the product of a Euclidean space and the Cantor set? Certainly, there is no C^{∞} diffeomorphism of the circle with such a minimal set. Furthermore, for C^{∞} diffeomorphisms of 2dimensional manifolds, the authors do not know such an example and it is plausible that there is no C^{∞} diffeomorphism with such a minimal set. However it is difficult to prove its non-existence because it is not easy to treat the condition that a minimal set is not locally homeomorphic to the product of a Euclidean space and the Cantor set. In this paper, we replace this condition by the local connectivity of a minimal set and examine the topological types of minimal sets for homeomorphisms of closed orientable surfaces. The condition of local connectivity appears in topological dynamics in a natural way either as a property of the space carrying the dynamics or as a property of minimal sets which is either assumed, proved or disproved. For example, the results of [14] show that a wide class of homogeneous flows admits no locally connected minimal sets, the paper [8] contains an example of a plane diffeomorphism which has a “pathological” minimal set which is nowhere locally connected while Kim [11] has shown that locally connected minimal sets of flows of compact separable metric spaces reduce to either single points or circles whenever they have cohomological dimension (with respect to Alexander-Spanier cohomology with coefficients in a principal ideal domain) ≤ 1.

Partially supported by the Polish KBN grant No. 2P03A03318^{†}, Grant-in-Aid for Scientific Research (No. 11554001^{‡}), Japan Society for the Promotion of Science, Japan, and European Commission contract No. ICA1-CT-2002-70017^{∗}.

Our results here begin with the following

Theorem 1. Let f be a homeomorphism of a closed orientable surface Σ different from the torus T^{2}. If a minimal set 𝔐 of f is locally connected, then 𝔐 is either a finite set or a finite disjoint union of simple closed curves.

Theorem 2. Let f be a homeomorphism of T^{2}. If there exists a locally connected minimal set 𝔐 which is neither finite, nor a finite disjoint union of simple closed curves, nor the whole T^{2}, then 𝔐 is the unique minimal set of f. This set 𝔐 satisfies the following conditions (1) ∼ (5), where {U_{i}}_{i=1,2,···} denotes the family of all the connected components for the complement of 𝔐 :

(1) each U_{i} is the interior of an embedded disc (i = 1, 2, ⋯),

(2) \{\overline{U_{i}}\}_{i=1,2,···} is a null sequence (i. e. the diameter of U_{i} tends to 0 as i → ∞),

(3) \overline{U_{i}} intersects \overline{U_{j}} at most at one point when i ≠ j, and their intersection (if non-empty) consists of a locally separating point of 𝔐,

(4) there is no finite chain U_{i_{1}}, U_{i_{2}}, ⋯, U_{i_{n}} (n > 1) such that \overline{U_{i_{j}}} ∩ \overline{U_{i_{j+1}}} ≠ ∅ (j = 1, 2, ⋯, n − 1) and \overline{U_{i_{1}}} ∩ \overline{U_{i_{n}}} ≠ ∅,

(5) 𝔐 is connected.

If, instead of conditions (3) and (4) of Theorem 2, we assume that \{\overline{U_{i}}\} consists of mutually disjoint sets, then 𝔐 appears to be homeomorphic to the Sierpiński T^{2}-set, which is obtained from T^{2} by removing the interiors of a null sequence of mutually disjoint closed discs whose union is dense in T^{2} (compare [2]). Thus we obtain the following:

Corollary 1. Let f be a homeomorphism of T^{2}. Any locally connected minimal set without a locally separating point either is finite, or coincides with the whole T^{2}, or consists of a finite disjoint union of simple closed curves, or is homeomorphic to the Sierpiński T^{2}-set.

The next (and last) result here shows that the assumption of absence of locally separating points cannot be deleted from Corollary 1.

Theorem 3. There exists a homeomorphism of T^{2} having a locally connected minimal set which admits a locally separating point and is not a finite disjoint union of simple closed curves.

In §1, we will construct a homeomorphism of T^{2} satisfying the condition of Theorem 3 by pinching holes of the Sierpiński T^{2}-set. Theorems 1 and 2 are proved in §3 and §4 respectively. In order to establish these theorems, we will show in §2 (Lemma 2) the non-existence of cut points in any connected minimal set.

Note that by results of Chu [3] the construction similar to that of Theorem 3 cannot be performed for flows (or actions of arbitrary connected topological groups).

1 Pinching Sierpiński T^{2}-sets

Let X be a compact metric space and S be a subset of X. As usually, we denote by ∂S the frontier of S and by int S its interior. Furthermore, diam S denotes the diameter of S, i. e. the smallest upper bound for the distances of points in S. A countable collection {S_{i}}_{i=1,2,···} of subsets S_{i} is called a null sequence if, for each ε > 0, only finitely many of the sets S_{i} have diameter greater than ε ([4]). In other words, lim_{i→∞} diam S_{i} = 0.

A point z of S is called a cut point of S if S ∖ {z} is not connected in S. Also, a point z of a subset S is called locally separating if there exists a connected neighbourhood U of z in S such that U ∖ {z} is not connected. Finally, let us recall that a subset S is locally connected if, for any point z of S and any neighbourhood U of z in S, one can find a connected neighbourhood of z contained in U.

Let f be a homeomorphism of X. A non-empty subset 𝔐 of X is called minimal if 𝔐 is closed, invariant under f (i. e. f(𝔐) = 𝔐) and minimal with respect to the inclusion among all non-empty closed f-invariant sets. By Zorn Lemma, any homeomorphism of a compact metric space has a minimal set. When the whole X is a minimal set, the homeomorphism f is called minimal. Then all its orbits are dense. Typical examples of minimal homeomorphisms of surfaces are minimal translations of the torus T^{2} defined as follows: Let α and β be irrational numbers such that α/β is also irrational. A homeomorphism f of T^{2} defined by

f(x, y) = (x + α, y + β)

for x, y ∈ ℝ/ℤ is minimal and called a minimal translation of T^{2}.

Whyburn ([15]) showed that the Sierpiński curve (called also Sierpiński carpet) can be characterized as a subset of the sphere S^{2} obtained by removing the interiors of a null sequence of mutually disjoint closed discs whose union is dense in S^{2}. His arguments can be also applied to such subsets of the torus T^{2} (and arbitrary closed manifolds, [2]). Thus we may define the Sierpiński T^{2}-set as a subset of T^{2} obtained by removing the interiors of a null sequence of mutually disjoint closed discs whose union is dense in T^{2}.

Aarts and Oversteegen ([1]) constructed a homeomorphism of the Sierpiński curve with a dense orbit. They inserted mutually disjoint discs into S^{2} and extended a homeomorphism of S^{2} with a dense orbit to a homeomorphism of S^{2} with the union of inserted discs invariant. This construction can be performed also in the case of a minimal translation of T^{2} (see [2] for a detailed description), so we can obtain in this way a homeomorphism f of T^{2} with a minimal set 𝔐 homeomorphic to the Sierpiński T^{2}-set and such that the family {f^{n}(U)}_{n∈ℤ} consists of mutually disjoint sets for any connected component U of T^{2} ∖𝔐. By suitable use of this homeomorphism, we will construct soon a homeomorphism of T^{2} satisfying the conditions of Theorem 3.

Remark 1. A C^{3−ε} diffeomorphism of T^{2} with a minimal Sierpiński T^{2}-set has been constructed by McSwiggen ([13]) for any ε > 0. To get it, he chooses an Anosov diffeomorphism of T^{3} and modifies in a suitable way the first return map of a global cross section of the unstable foliation.

Proof of Theorem 3. Let f be the mentioned above homeomorphism of T^{2} with a minimal set 𝔐 homeomorphic to the Sierpiński T^{2}-set and such that the sets f^{n}(U), n ∈ ℤ, are mutually disjoint for any connected component U of T^{2} ∖ 𝔐.

Let {U_{i}}_{i=1,2,···} denote the family of all the connected components of T^{2}∖𝔐. Let us choose a properly embedded (i. e., such that the intersection l∩∂U_{1} coincides with the pair of end points of l) arc l contained in \overline{U_{1}}. Since \{\overline{U_{i}}\}_{i=1,2,···} is a null-sequence, diam f^{n}(l) converges to 0 as n → ±∞. Let us define the equivalence relation ∼ by z_{1} ∼ z_{2} (z_{1}, z_{2} ∈ T^{2}) whenever either z_{1} = z_{2} or both, z_{1} and z_{2}, are contained in f^{n}(l) for some n ∈ ℤ (Figure 1). Let π : T^{2} → T^{2}/ ∼ denote the quotient map. The family of the closed sets {f^{n}(l)} and the points of T^{2} − ⋃_{n∈ℤ} f^{n}(l) forms so called decomposition with respect to {f^{n}(l)}_{n∈ℤ}. This decomposition is shrinkable (in the sense of [4], see also [2]), and therefore T^{2}/ ∼ is homeomorphic to T^{2} by Theorem 6 in [4], p. 28. (Certainly, π itself is not a homeomorphism but just a near homeomorphism, i.e. it can be approximated by homeomorphisms in the sense described in [4].) Let us define a homeomorphism g of T^{2}/ ∼ by g(π(z)) = πf(z).

f  f  g  g

Figure 1: Pinching process

We will show that π(𝔐) is a minimal set of g. Suppose that K is a non-empty closed g-invariant set contained in π(𝔐). Then π^{−1}(K) ∩ 𝔐 is closed, f-invariant and contained in 𝔐. Thus the set π^{−1}(K) ∩ 𝔐 is either empty or coincides with 𝔐. In the first case, π^{−1}(K) would be contained in ⋃_{i=1}^{∞} U_{i}. Since the sets f^{n}(U_{i}) are mutually disjoint, the ω-limit set of a point of π^{−1}(K) would be disjoint from ⋃_{i=1}^{∞} U_{i}, a contradiction. Thus π^{−1}(K) ∩ 𝔐 coincides with 𝔐, and π^{−1}(K) contains 𝔐. Therefore, K = π(π^{−1}(K)) coincides with π(𝔐) and this implies that π(𝔐) is a minimal set of g indeed.

Next, we show that π(𝔐) is locally connected. Let p be a point of π(𝔐) and U a neighbourhood of p in T^{2}. First, suppose that p does not belong to π(⋃_{n∈ℤ} f^{n}(l)). Let q denote the unique point such that π(q) = p. Since 𝔐 is locally connected, there is a neighbourhood V of q in T^{2} such that the intersection V ∩ 𝔐 is connected and contained in π^{−1}(U) ∩ 𝔐. By one of the properties of Sierpiński T^{2}-sets ([15]), ∂V can be further assumed to be disjoint from ⋃_{n∈ℤ} f^{n}(l). Then π^{−1}π(V) is equal to V. In fact, if r is a point of π^{−1}π(V), then π(r) lies in π(V), and there is a point z of V such that π(r) = π(z). If r = z, then obviously r lies in V. On the other hand, if r ≠ z, then there is n ∈ ℤ such that both, r and z, belong to f^{n}(l). Since f^{n}(l) is disjoint from ∂V, r belongs also to V. Thus the set π^{−1}π(V) is contained in V and, consequently, π^{−1}π(V) = V. Therefore, π(V) is open in T^{2}. Moreover, π(V) ∩ π(𝔐) coincides with π(V ∩ 𝔐) because of the following: if z_{1} ∈ V and z_{2} ∈ 𝔐 are such that π(z_{1}) = π(z_{2}) and z_{1} ≠ z_{2}, then there is n ∈ ℤ such that both, z_{1} and z_{2}, belong to f^{n}(l), and hence z_{2} lies in V. This implies the required equality π(V) ∩ π(𝔐) = π(V ∩ 𝔐). Thus π(V) ∩ π(𝔐) is a connected neighbourhood of p in π(𝔐), which is contained in U. Next, consider the case when p is a point of π(⋃_{n∈ℤ} f^{n}(l)). Let j denote the integer such that p is contained in π(f^{j}(l)), and q_{1} and q_{2} – the end points of f^{j}(l). We can choose neighbourhoods V_{i} (i = 1, 2) of q_{i} in T^{2} such that V_{i} ∩ 𝔐 is contained in π^{−1}(U) ∩ 𝔐, V_{i} ∩ 𝔐 is connected and ∂V_{i} ∩ (⋃_{n≠j} f^{n}(l)) = ∅. Let W = V_{1} ∪ V_{2}. Then π(W) ∩ π(𝔐)(= ⋃_{i=1}^{2} π(V_{i} ∩ 𝔐)) is a connected neighbourhood of p in π(𝔐) contained in U by the same reason as above. This shows that the set π(𝔐) is locally connected indeed.

Finally, we shall show that our set π(𝔐) has a locally separating point. Let z_{1} and z_{2} denote the end points of l. For any i = 1, 2, there exists a neighbourhood V_{i} of z_{i} in T^{2} such that V_{i}∩𝔐 is connected, ∂V_{i}∩(⋃_{n≠0} f^{n}(l)) = ∅ and V_{1}∩V_{2}∩𝔐 = ∅. Therefore, π(V_{1} ∪ V_{2}) ∩ π(𝔐) is a connected neighbourhood of p = π(l) in π(𝔐) (by the same argument as that in the proof of local connectedness of π(𝔐)). Moreover, π(V_{1}) ∖ {p} and π(V_{2}) ∖ {p} are disjoint open subsets of π(𝔐). Therefore, π(l) separates 𝔐 locally. ■

Remark 2. A point which is not contained in π(⋃_{n∈ℤ} f^{n}(l)) is not locally separating. Thus the minimal set π(𝔐) is a locally connected continuum (i. e. a compact connected set) which is not homogeneous and admits a minimal homeomorphism. Another one-dimensional continuum which is not homogeneous and admits a minimal homeomorphism was introduced in Theorem 14.24 in [7]; that continuum is not locally connected (compare also [6]).

Remark 3. In the proof of Theorem 3, we inserted just one properly embedded arc into the closure of a connected component of the minimal set. We can modify this construction easily by inserting a null-sequence of infinitely many pairwise disjoint properly embedded arcs in there.

Remark 4. Aarts and Oversteegen ([1]) showed that the Sierpiński curve admits no minimal homeomorphism while Kato ([9]) proved that the Sierpiński curve admits no expansive homeomorphism (compare [1] again). Our article is in fact strongly stimulated by these papers.

2 Cut points of minimal sets

In this section, we provide some general properties of minimal sets for homeomorphisms of arbitrary compact metric spaces. Although there exists a compact metric space which is not homogeneous but admits a minimal homeomorphism (see Remark 2 in §1), such minimal sets enjoy ‘homogeneity’ of certain kind.

Throughout the paper, the following simple observation will be used for several times.

Lemma 1. Let 𝔐 be a connected minimal set of a homeomorphism f of a compact metric space. Then there is no non-empty compact proper subset K of 𝔐 such that K, f(K), ⋯, f^{n}(K) (n ≧ 0) are mutually disjoint and either f^{n+1}(K) is contained in some f^{m}(K) (0 ≦ m ≦ n) or f^{n+1}(K) contains some f^{m}(K) (0 ≦ m ≦ n).

Proof. First, we consider the case when f^{n+1}(K) is contained in some f^{m}(K) (0 ≦ m ≦ n). If such a compact set K exists, then its ω-limit set ω(K) = ⋂_{k≧0} \overline{⋃_{j≧k} f^{j}(K)} is compact, f-invariant and contains ⋂_{j≧0} f^{(n+1−m)j+m}(K) (≠ ∅). Hence 𝔐 coincides with ω(K). On the other hand, ω(K) is contained in K ∪ f(K) ∪ ⋯ ∪ f^{n}(K), which is also contained in 𝔐. Thus we have 𝔐 = K ∪f(K)∪⋯∪f^{n}(K). However, this contradicts the assumption K ≠ 𝔐 when n = 0 and that of connectedness of 𝔐 when n > 0.

One can complete the proof by replacing f with f^{−1} and f^{n+1}(K) with K in the case when f^{n+1}(K) contains some f^{m}(K) (0 ≦ m ≦ n) . ■

Lemma 2. Let 𝔐 be a connected minimal set of a homeomorphism f of a compact metric space X. Then 𝔐 has no cut points.

Proof. Assume that 𝔐 has a cut point z. Certainly, each of the points f^{n}(z), n ∈ ℤ, cuts 𝔐 as well. By definition, 𝔐 ∖ {z} consists of two non-empty sets V_{1} and V_{2} such that both of them are open in 𝔐. Let K_{i} (i = 1, 2) denote V_{i} ∪ {z}. Then K_{i}’s (i = 1, 2) are closed in 𝔐 (just because V_{i}’s are open). Furthermore, K_{i}’s are connected. In fact, if one of them, say K_{1}, were not connected, then there would exist two disjoint closed subsets A_{1} and A_{2} of 𝔐 such that K_{1} = A_{1}∪A_{2} and z ∈ A_{1}. Then the sets A_{1} ∪ K_{2} and A_{2} would be disjoint and closed in 𝔐 contradicting the connectedness of 𝔐. Thus we have two continua K_{1} and K_{2} contained in 𝔐 and such that K_{1} ∩ K_{2} consists of the single point z, K_{1} ∪ K_{2} = 𝔐, and both, K_{1} and K_{2}, contain more than one point.

We claim that either f(K_{1}) ∩ K_{1} = ∅ or f(K_{2}) ∩ K_{2} = ∅. Since 𝔐 contains at least three points, z is not fixed by f. Hence z lies in the union of the sets f(K_{1} ∖ {z}) and f(K_{2} ∖ {z}), and therefore it does not belong to the intersection of their complements f(K_{2}) and f(K_{1}). First, suppose that z ∉ f(K_{2}). Then f(K_{2})∩K_{1} and f(K_{2})∩K_{2} are disjoint closed sets. Since f(K_{2}) is connected, f(K_{2}) is contained in either K_{1} ∖ {z} or K_{2} ∖ {z}. The second possibility is excluded by Lemma 1. Thus f(K_{2}) ⊂ K_{1} ∖ {z}, and hence f(K_{2}) ∩ K_{2} = ∅. In the same way, f(K_{1}) ∩ K_{1} = ∅ if f(K_{1}) does not contain z. Thus one can always find i ∈ {1, 2} for which f(K_{i}) ∩ K_{i} = ∅.

Now, using the same argument as above inductively, we will show that all the sets K_{i}, f(K_{i}), f^{2}(K_{i}), ⋯ are mutually disjoint. Suppose that K_{i}, f(K_{i}), ⋯, f^{n}(K_{i}) (n ≧ 1) are mutually disjoint but f^{n+1}(K_{i}) intersects f^{m}(K_{i}) for some m (0 ≦ m ≦ n). If f^{n+1}(K_{i}) does not contain the cut point f^{m}(z), then the connected set f^{n+1}(K_{i}) is contained in f^{m}(K_{i}), what contradicts Lemma 1. Thus the point f^{m}(z) is contained in f^{n+1}(K_{i}). Since z is not a periodic point, f^{m}(z) has to lie in f^{n+1}(K_{i} ∖ {z}). Let K_{j} denote the other of our two continua in 𝔐 (i. e. K_{j} = (𝔐 ∖ K_{i}) ∪ {z}). Then f^{n+1}(K_{j}) does not contain f^{m}(z). Thus the set f^{n+1}(K_{j}) is contained either in f^{m}(K_{i} ∖ {z}) or in f^{m}(K_{j} ∖ {z}). In the second case, f^{n+1}(K_{i}) contains f^{m}(K_{i}), what contradicts Lemma 1. Thus f^{n+1}(K_{j}) has to be contained in f^{m}(K_{i} ∖ {z}). If m > 0, then f^{n+1}(K_{j} ∖ {z}) ∩ K_{i} = ∅ just because K_{i} ∩ f^{m}(K_{i}) = ∅. Thus f^{n+1}(K_{i}) contains K_{i}. This contradicts Lemma 1 again. On the other hand, if m = 0, then f^{n+1}(K_{j} ∖ {z}) is contained in K_{i} ∖ {z}. Since f(K_{i}) is disjoint from K_{i}, we obtain that f^{n+1}(K_{j} ∖ {z}) ∩ f(K_{i}) = ∅. This implies that f^{n}(K_{j} ∖ {z}) ∩ K_{i} = ∅, and hence f^{n}(K_{i}) contains K_{i}, what contradicts the assumption. Thus all the sets K_{i}, f(K_{i}), f^{2}(K_{i}), ⋯ are mutually disjoint indeed. (Let us remark that all the family {f^{k}(K_{i})}_{k∈ℤ} consists of mutually disjoint sets just because f^{k_{1}}(K_{i}) ∩ f^{k_{2}}(K_{i}) = f^{k_{1}}(K_{i} ∩ f^{k_{2}−k_{1}}(K_{i})) = ∅ if k_{2} > k_{1}.)

Since f^{k}(K_{i} ∖ {z}) (k ∈ ℤ) are non-empty open sets, the complement of the union ⋃_{k∈ℤ} f^{k}(K_{i} ∖ {z}) is f-invariant, closed and different from 𝔐, and hence it has to be empty. In other words, the family A = {f^{k}(K_{i} ∖{z})}_{k∈ℤ} covers 𝔐. Since 𝔐 is compact, some finite subfamily of A covers 𝔐. Since, as was observed before, the sets f^{k}(K_{i} − z), k ∈ ℤ are mutually disjoint, this contradicts the assumption that 𝔐 is connected. ■

Next we provide some properties of locally connected continua without cut points. These will be used frequently in the proofs of Theorems 1 and 2 in §3 and §4.

Lemma 3. Let X be a compact metric space and 𝔐 – a connected and locally connected closed subset of X without cut points. If there exist a compact subset K of 𝔐 and an arc l_{1} in 𝔐 such that K contains at least two points and K ∩l_{1} consists of a single point z, then there is an arc l_{2} in 𝔐 such that one of its end points is z, the other end point w of l_{2} is contained in K ∖ {z}, and l_{2} ∖ {z, w} is disjoint from K (Figure 2).

Hereafter, by an arc we mean an injective continuous image of a closed interval.

K l_{1} x l_{2} z w Figure 2:

Proof. Let x denote the end point of l_{1} different from z. Denote by V_{1} the pathconnected component of 𝔐 ∖ {z} containing x. Let V_{2} = 𝔐 − (V_{1} ∪ z). Then V_{1} ∩ V_{2} = ∅ and V_{1} ∪ V_{2} = 𝔐 ∖ {z}, by definition. By Mazurkiewicz-Moore-Menger Theorem ([12], p. 254), any complete locally connected metric space is locally arcwise connected, therefore our set 𝔐 is locally arcwise connected (and arcwise connected too). Thus V_{1} and V_{2} are open in 𝔐 ∖ {z}. By Lemma 2, the set V_{2} has to be empty. Let us choose an arbitrary point p of K, p ≠ z. Certainly, p belongs to V_{1}. Thus we can find an arc l_{3} joining points p and x and contained in 𝔐 ∖ {z}. Denote by β the connected component of l_{1} ∖ l_{3} containing z. The end point q of β, q ≠ z, is connected with a point of K by a subarc l_{4} of l_{3} which intersects K only at its end points. Let l_{2} denote the union of β and l_{4}. The arc l_{2} satisfies the conditions of Lemma 3. ■

Corollary 2. Let f be a homeomorphism of a compact metric space. If a minimal set 𝔐 is connected and locally connected and is not a single point, then 𝔐 contains a simple closed curve.

Proof. Let x and y be two distinct points of 𝔐, l – an arc joining x and y in 𝔐 and z – a point of l different from x and y. Denote by l_{1} the subarc of l between z and x, and by K the subarc of l between z and y. There exists an arc l_{2} in 𝔐 satisfying the conditions of Lemma 3. The union l ∪ l_{2} contains a simple closed curve (which is obviously contained in 𝔐). ■

3 Complements of minimal sets

In order to consider locally connected minimal sets, it is important to examine topological properties of their complements. In this section, we will prove some facts (Lemmas 4 and 5) concerning simple closed curves in the boundary of such complements. These facts will be used to prove Theorem 1 in the final part of this section.

Lemma 4. Let f be a homeomorphism of a closed orientable surface Σ with a connected and locally connected minimal set 𝔐. Let U be a connected component of Σ − 𝔐. If there exists a simple closed curve C contained in the frontier ∂U of U and satisfying the following conditions (Figure 3):

(1) Σ − C consists of two disjoint connected open sets V_{1} and V_{2},

(2) V_{1} contains U, and

(3) V_{2} is disjoint from 𝔐,

then 𝔐 coincides with C. (In particular, U = V_{1}.)

V_{2}  C  l_{1} (⊂ 𝔐) V_{1} U Figure 3:

Proof. Assume that 𝔐 does not coincide with C. Let p be a point of 𝔐 ∩ V_{1}. Since 𝔐 is connected, there exists an arc γ_{1} of 𝔐 joining p and a point q of C such that γ_{1} ∖ {q} is disjoint from C. By Lemma 3, one can find a properly embedded arc l_{1} of \overline{V_{1}} contained in 𝔐.

We claim that V_{1} ∖l_{1} is connected. In fact, if V_{1} ∖l_{1} consists of two disjoint open sets W_{1} and W_{2}, then both C ∩\overline{W_{1}} and C ∩\overline{W_{2}} are non-empty arcs with common end points (just because the end points of l_{1} cut C into two arcs and V_{1} is connected). On the other hand, either W_{1} or W_{2}, say W_{1} contains U. Then C is contained in \overline{W_{1}} because C ⊂ ∂U. However this contradicts the condition C ∩ \overline{W_{2}} ≠ ∅. Thus V_{1} ∖ l_{1} is connected indeed.

Let q and q′ denote the end points of l_{1} and r be a point of C different from q and q′. Since 𝔐 is a minimal set, the orbit starting from q accumulates at r. Furthermore, q is a branch point of C ∪ l_{1}. Thus the image of a neighbourhood of q in 𝔐 by this orbit cannot be contained in C. Therefore, condition (3) of our Lemma implies that, arbitrarily close to r, there exists a point of 𝔐, which lies in V_{1} but not in C. Since 𝔐 is locally connected, there exists a small arc close to r and contained in 𝔐 which intersects C only at one of its end points. Applying Lemma 3 we get an arc l_{2} contained in 𝔐 and such that l_{2} intersects C ∪ l_{1} only at its end points and one of the end points is contained in C. Then V_{1} ∖ (l_{1} ∪ l_{2}) is connected because, if not, two sides of l_{2} would be contained in distinct connected components of V_{1} ∖ (l_{1} ∪ l_{2}) one of which contains U, therefore ∂U could not contain C as above.

Proceeding inductively, we obtain infinitely many arcs l_{1}, l_{2}, … in 𝔐 such that l_{i} intersects C ∪ l_{1} ∪ l_{2} ∪ ⋯ ∪ l_{i−1} only at its end points and V_{1} ∖ (l_{1} ∪ l_{2} ∪ ⋯ ∪ l_{i}) is connected for all i = 1, 2, · · ·.

Finally, choose a regular neighbourhood R of C∪(l_{1}∪l_{2}∪⋯∪l_{i}). Let Σ_{1} = \overline{V_{2} ∪ R} and Σ_{2} = \overline{Σ − Σ_{1}}. Then the Euler characteristic χ(Σ_{2}) of Σ_{2} is smaller than or equal to 2 (just because Σ_{2} is connected). The Mayer-Vietoris sequence yields

χ(Σ_{1}) = χ(Σ_{1} ∪ Σ_{2}) − χ(Σ_{2}) ≧ (2 − 2g) − 2 ≧ −2g,

where g is the genus of Σ. On the other hand,

χ(Σ_{1}) ≤ 1 − i.

Since this is impossible for a sufficiently large i, 𝔐 coincides with C. ■

In the case when C does not separate Σ, we need some additional consideration because U may accumulate at C from both sides.

Lemma 5. Let again f be a homeomorphism of a closed orientable surface Σ with a connected and locally connected minimal set 𝔐. Let U be a connected component of Σ ∖ 𝔐. If ∂U coincides with 𝔐 and there exists a simple closed curve C contained in ∂U such that Σ ∖ C is connected (Figure 4), then 𝔐 coincides with C.

Proof. Assume that 𝔐 does not coincide with C. As before, we will construct by induction infinitely many arcs l_{1}, l_{2}, ⋯ in 𝔐 such that l_{i} intersects C ∪ l_{1} ∪ ⋯ ∪ l_{i−1} at its end points and Σ ∖ (C ∪ l_{1} ∪ ⋯ ∪ l_{i}) is connected for i = 1, 2, · · ·.

The first step of induction (existence of l_{1}) follows from Lemma 3 as in the proof of Lemma 4.

Suppose that l_{1}, l_{2}, …, l_{n} (n ≥ 1) satisfy the above conditions. Let S_{n} denote C ∪ l_{1} ∪ ⋯ ∪ l_{n}. Since S_{n} has finitely many branch points, there is an arc l ⊂ 𝔐 such that l intersects S_{n} only at its end points and one of the end points of l is contained in C − (l_{1} ∪ ⋯ ∪ l_{n}). Suppose that Σ − (S_{n} ∪ l) is not connected. Let V_{1} and V_{2} denote the connected components of Σ − (S_{n} ∪ l) such that U is contained in V_{1}. Then ∂U (= 𝔐) is contained in \overline{V_{1}}, and hence V_{2} is disjoint from 𝔐. Since

U C 𝔐 Figure 4:

l intersects C at C − (l_{1} ∪ ⋯ ∪ l_{n}), the set (C ∩ \overline{V_{2}}) − (l_{1} ∪ l_{2} ∪ ⋯ ∪ l_{n}) contains a non-empty open arc α. Then a one-sided neighbourhood of α in V_{2} is disjoint from U. Next, let us choose another arc l′ of 𝔐 such that l′ intersects S_{n} only at its end points, one of them being contained in α. Since V_{2} is disjoint from 𝔐, l′ is disjoint from V_{2}. Suppose also that Σ − (S_{n} ∪ l′) is not connected. Let V′_{1} and V′_{2} denote the connected components of Σ − (S_{n} ∪ l′) such that U is contained in V′_{1}. Then V′_{2} is disjoint from U as above. By the same argument as for l, C ∩ \overline{V′_{2}} contains a non-empty open arc α′ contained in α such that one of the end points of α′ is an end point of l′. Since l′ is disjoint from V_{2}, both sides of α′ are disjoint from U. Since this contradicts the assumption that C is contained in ∂U, we obtain that either Σ ∖ (S_{n} ∪ l) or Σ ∖ (S_{n} ∪ l′) is connected so we can put either l_{n+1} = l (in the first case) or l_{n+1} = l′ (in the second one). As promised, induction provides infinitely many arcs l_{1}, l_{2}, … such that each l_{i} intersects C ∪ l_{1} ∪ ⋯ ∪ l_{i−1} at end points and all the sets Σ ∖ (C ∪ l_{1} ∪ ⋯ l_{i}) are connected.

One can complete the proof by the same arguments as those in the final step of the proof of Lemma 4. ■

Proof of Theorem 1. Let 𝔑 be a connected component of the minimal set 𝔐. Certainly, any connected component of a locally connected space is open. Hence the complement of ⋃_{n∈ℤ} f^{n}(𝔑) is an f-invariant closed set, and has to be empty. In other words, {f^{n}(𝔑)}_{n∈ℤ} is an open covering of 𝔐. By the compactness of 𝔐, 𝔐 coincides with the union of a finite subfamily of {f^{n}(𝔑)}_{n∈ℤ}. Consequently, there exists j ∈ ℤ such that the equality f^{j}(𝔑) = 𝔑 holds. We assume that j_{0} is the least positive integer j satisfying this equality. By the minimality of 𝔐, 𝔐 coincides with ⋃_{n=0}^{j_{0}−1} f^{n}(𝔑), and 𝔑 is a minimal set of f^{j_{0}}.

By assumption, the Euler characteristic of Σ is different from zero. Hence, f has a periodic point (see, for example, [10], p. 330, Exercise 8.6.2), denoted here by p. Let m denote its period. Then p is a fixed point of f^{mj_{0}}. Since 𝔑 is connected and minimal for f^{j_{0}}, 𝔑 is – by Theorem 2.28 in [7] – minimal also for f^{mj_{0}}. Thus we have only to show that any connected and locally connected minimal set 𝔐 of a homeomorphism f of a closed orientable surface Σ with a fixed point p coincides with either a single point or a simple closed curve.

Suppose that 𝔐 is not a single point. Then p cannot not belong to 𝔐. Let U be a connected component of Σ ∖ 𝔐 containing p. Then f(U) coincides with U, and hence f(∂U) coincides with ∂U. Since ∂U is a closed invariant set, our minimal set 𝔐 coincides with ∂U too. By Corollary 2, 𝔐 contains a simple closed curve C. If C does not separate Σ, then 𝔐 coincides with C by Lemma 5. If C separates Σ, i. e. Σ ∖ C is the union of two disjoint open sets V_{1} and V_{2} (without loss of generality, we may suppose that U is contained in V_{1}), then V_{2} is disjoint from 𝔐 (just because 𝔐 coincides with ∂U (⊂ \overline{V_{1}})), and again 𝔐 coincides with C, this time by Lemma 4. ■

4 Homeomorphisms of the torus

Let f be a homeomorphism of T^{2} and U – a connected component of the complement of its minimal set 𝔐. If f(U) coincides with U, then ∂U is closed and f-invariant, therefore ∂U coincides with 𝔐. Corollary 2 obtained in the course of proof of Theorem 1 yields the existence of a simple closed curve contained in ∂U in this case. In the case when the sets f^{n}(U), n ∈ ℤ, are mutually disjoint, this argument does not work. In order to find a simple closed curve in ∂U, we need several preparatory facts (Lemmas 6, 7 and 8).

Lemma 6. Let f be a homeomorphism of T^{2} and let 𝔐 be a connected and locally connected minimal set which is neither a single point nor a simple closed curve. Then, for any connected component U of T^{2}−𝔐, its saturation {f^{n}(U)}_{n∈ℤ} consists of mutually disjoint sets.

Proof. Let us assume on the contrary that the sets {f^{n}(U)}_{n∈ℤ} are not mutually distinct. Then there is k ≠ 0 such that f^{k}(U) = U. Since 𝔐 is connected, 𝔐 is also a minimal set of f^{k}. Now ∂U is a closed set invariant under f^{k}. Thus ∂U = 𝔐. By the same arguments as that for Theorem 1, 𝔐 coincides with a simple closed curve. This contradicts the assumption. ■

Lemma 7. Let, as before, f be a homeomorphism of T^{2} and let 𝔐 be a connected and locally connected minimal set, which is neither a single point nor a simple closed curve nor the whole T^{2}. Let {U_{i}}_{i=1,2,···} denote the family of all the connected components of T^{2} ∖ 𝔐. Then \{\overline{U_{i}}\}_{i=1,2,···} is a null sequence.

Proof. First, let us remark that (by Lemma 6) T^{2}∖𝔐 has infinitely many connected components. Assume that \{\overline{U_{j}}\}_{j=1,2,···} is not a null sequence. Then there exists ε > 0 such that infinitely many of \overline{U_{j}}’s have diameter greater than ε. Denote by \{V_{i}\}_{i=1,2,···} the collection of all such U_{j}’s. Denote by d the standard metric on T^{2} and, for each i ∈ ℕ, choose points x_{i} and y_{i} of V_{i} such that d(x_{i}, y_{i}) > ε, and an arc γ_{i} in V_{i} joining x_{i} and y_{i}. Let z_{i} be a point of γ_{i} such that d(x_{i}, z_{i}) > ε/3 and d(y_{i}, z_{i}) > ε/3. Passing several times to a subsequence (if necessary), we may assume that {x_{i}}, {y_{i}} and {z_{i}} converge (as i → ∞) to points x, y and z, respectively (Figure 5). Then we

z_{i} x_{i}  y_{i} γ_{i} V_{i} V_{i+1} x_{i+1}  y_{i+1} z_{i+1} x  z  y Figure 5:

have d(x, z) ≧ ε/3 and d(y, z) ≧ ε/3. Furthermore, z is contained in 𝔐 because, if not, z would belong to some connected component U_{k} of T^{2} ∖ 𝔐, and – since U_{k} contains at most one point of the sequence {z_{i}} – this sequence would not be able to converge to z. Since {x_{i}} and {y_{i}} converge to x and y respectively, there is N > 0 such that d(x_{i}, x) < ε/6 and d(y_{i}, y) < ε/6 if only i ≧ N. Let

D = {w; d(w, z) ≦ ε/6}.

The points x_{i} and y_{i} do not lie in D when i ≧ N. By the local connectivity of 𝔐, there exists a neighbourhood W of z in T^{2} such that W is contained in D and W ∩𝔐 is path-connected. Replacing eventually W by its connected component containing z we may assume that W itself is connected as well. Let L be an integer greater than N and such that z_{i} lies in W whenever i ≧ L. For j = L, L + 1, L + 2, the points x_{j} and y_{j} are not contained in D, and hence we can choose properly embedded arcs β_{j} contained in D ∩ γ_{j} and passing through z_{j}. The arcs β_{L}, β_{L+1}, β_{L+2} split D into four closed discs. Among these discs, there are two whose boundaries consist of two arcs chosen from {β_{L}, β_{L+1}, β_{L+2}} and two others contained in ∂D. Since z does not belong to the boundaries of these two discs, one of them (denoted by Δ from now) does not contain z. Without loss of generality, we can assume that ∂Δ consists of β_{L}, β_{L+1} and two arcs contained in ∂D, and furthermore that β_{L+1} is closer to z than β_{L} (i. e., z and β_{L} are contained in different components of D ∖ β_{L+1}).

We claim that W ∩ 𝔐 ∩ int Δ ≠ ∅. Indeed, since W is connected, there exists an arc α_{1} in W joining z and z_{L}. Then α_{1} intersects β_{L+1}. Thus there exists a subarc α_{2} of α_{1} properly embedded in Δ and such that one of the end points is contained in β_{L} while the other one in β_{L+1}. Since β_{L} ⊂ V_{L} and β_{L+1} ∩ V_{L} = ∅, one can find a point q of α_{2} contained in the intersection of ∂V_{L} and int Δ.

Since q ∈ 𝔐 ∩ W, q can be connected to z by a path l in 𝔐 ∩ W. However, this is impossible since such an l would intersect β_{L+1} which is disjoint from 𝔐 ∩ W. Therefore, \{\overline{U_{i}}\}_{i=1,2,···} is a null sequence indeed. ■

Any disc is contained in a Janiszewski space without a cut point, which is homeomorphic to S^{2} ([12]). By Theorem 4 in [12], §61,II, we have the following.

Lemma 8. Let X be a free of cut points and locally connected continuum contained in a disc D and such that ∂D is contained in X. Let U be a connected component of D ∖ X. Then U is the interior of a disc.

Lemma 9. Let f be a homeomorphism of T^{2} and U – a connected component of the complement of a connected and locally connected minimal set 𝔐 of f. If 𝔐 is neither a single point nor a simple closed curve, then U is the interior of a disc.

Proof. By Corollary 2, 𝔐 contains a simple closed curve C. By Lemma 3, there is an arc l_{1} of T^{2} which is contained in 𝔐 and intersects C only at its end points. Then the θ-curve (compare [12], p. 328) C ∪ l_{1} of T^{2} belongs to one of the three types according to the number of the connected components of its complement (which is always smaller than or equal to three).

First, we consider the case when T^{2} ∖ (C ∪ l) is connected. Then the manifold obtained by cutting T^{2} along C ∪ l becomes a closed disc D_{1} after pasting a circle to T^{2} ∖ (C ∪ l) along the boundary. Since C ∪ l is not the whole 𝔐, int D_{1} contains a point p of 𝔐. Let d_{1} denote the distance between ∂D_{1} and p. By the minimality of 𝔐, for any point q of ∂U, there is n ∈ ℤ such that the distance between f^{n}(q) and p is smaller than d_{1}/3. Furthermore, by Lemmas 6 and 7, we may assume that diam f^{n}(U) is also smaller than d_{1}/3. Hence f^{n}(U) is contained in the interior of D_{1}. Thus f^{n}(U) is a connected component of D_{1} ∖ 𝔐 such that \overline{f^{n}(U)} is disjoint from ∂D_{1}. By Lemma 8, there is a disc D_{2} in int D_{1} such that int D_{2} = f^{n}(U). Thus U is the interior of the disc f^{−n}(D_{2}) ⊂ T^{2}.

Next, we assume that T^{2} ∖ (C ∪ l) has three connected components. Then the manifold obtained by cutting T^{2} along C ∪ l consists of two discs Σ_{1} and Σ_{2} and a one-punctured torus Σ_{3}. The discs Σ_{1} and Σ_{2} are adjacent by an arc. If the interiors int Σ_{1} and int Σ_{2} are disjoint from 𝔐, then the intersection Σ_{1} ∩Σ_{2} contains a closed arc α “isolated” in 𝔐 (in the sense, that 𝔐 is locally homeomorphic to an arc in a neighbourhood of any point of α different from end points). By the minimality of 𝔐, {f^{n}(α)}_{n∈ℤ} covers 𝔐. Furthermore, by the compactness of 𝔐, finitely many of the sets f^{n}(α), n ∈ ℤ cover 𝔐. Therefore, 𝔐 is a simple closed curve, which contradicts the assumption. Thus either int Σ_{1} or int Σ_{2} contains a point p of 𝔐. By the same argument as in the case when T^{2} ∖ (C ∪ l) is connected, we can show that there exists n ∈ ℤ such that f^{n}(U) is contained in either Σ_{1} or Σ_{2}, so that \overline{f^{n}(U)} ∩ (C ∪ l) = ∅, and hence U is the interior of a disc as above.

Finally, we consider the case when T^{2} ∖ (C ∪ l) has two connected components. Then the manifold obtained by cutting T^{2} along C ∪ l consists of a disc Σ_{1} and an annulus Σ_{2}. Replacing C by ∂Σ_{1} if necessary, we may assume that C itself bounds Σ_{1}. If int Σ_{1} intersects 𝔐, then U is the interior of a disc by the same argument as in the case when T^{2} − (C ∪ l) is connected. Thus we may assume that int Σ_{1} is disjoint from 𝔐. The orbit starting from a branch point of C ∪ l accumulates at a point of C ∖ l, and there is a small arc in Σ_{2} such that one of the end points is contained in C ∖ l. By Lemma 3, there is an arc l′ of 𝔐 such that l′ intersects C ∪ l only at its end points, one of them being contained in C ∖ l. If Σ_{2} ∖ (l ∪ l′) is connected, then the manifold obtained by cutting Σ_{2} along l ∪ l′ is a disc adjacent to Σ_{1} (Figure 6 (a)). On the other hand, if Σ_{2} ∖ (l ∪ l′) is not connected, then the

l  l′ CΣ_{1}  Σ_{2} (a)  l l′ CΣ_{1}  Σ_{2} (b)

Figure 6:

manifold obtained by cutting Σ_{2} along l ∪ l′ is the union of a disc and an annulus (Figure 6 (b)). Here this disc is also adjacent to the disc Σ_{1}. In this case, U is the interior of a disc according to the same argument as in the case when T^{2} ∖ (C ∪ l) has three connected components. ■

Lemma 10. Let f be a homeomorphism of T^{2} and 𝔐 be a connected and locally connected minimal set of f. Let {U_{i}}_{i=1,2,···} denote the family of the connected components for the complement of 𝔐. If 𝔐 is neither a single point nor a simple closed curve nor the whole T^{2}, then

(1) \overline{U_{i}} intersects \overline{U_{j}} at most at one point when i ≠ j, and the intersection \overline{U_{i}} ∩ \overline{U_{j}} consists of a locally separating point of 𝔐 (if non-empty);

(2) there is no finite chain U_{i_{1}}, U_{i_{2}}, …, U_{i_{n}} (n > 1) such that \overline{U_{i_{j}}} ∩ \overline{U_{i_{j+1}}} ≠ ∅ (j = 1, 2, …, n − 1) and \overline{U_{i_{1}}} ∩ \overline{U_{i_{n}}} ≠ ∅.

Proof. First, we will show that \overline{U_{i}} intersects \overline{U_{j}} at most at one point when i ≠ j. Assume that \overline{U_{i}} ∩ \overline{U_{j}} contains two points p_{1} and p_{2}. Since \overline{U_{i}} and \overline{U_{j}} are discs, there is an arc γ_{1} (resp., γ_{2}) contained in \overline{U_{i}} (resp., \overline{U_{j}}) such that both γ_{1} and γ_{2} join p_{1} and p_{2} and, furthermore, γ_{1} (resp., γ_{2}) intersects ∂U_{i} (resp., ∂U_{j}) only at its end points. Since U_{i} ∩ U_{j} = ∅, the union γ_{1} ∪ γ_{2} is a simple closed curve, denoted hereafter by C. By Lemmas 6 and 7, diam f^{n}(\overline{U_{i}}) and diam f^{n}(\overline{U_{j}}) converge to 0 as n → ∞. Thus there is N > 0 such that f^{N}(C) bounds a disc D_{1}. Let z_{l} (l = 1, 2) be a point of γ_{l} different from p_{1} and p_{2}. Since f^{N}(z_{1}) (resp., f^{N}(z_{2})) is a point of f^{N}(U_{i}) (resp., f^{N}(U_{j})), the interior int D_{1} intersects f^{N}(U_{i}) and f^{N}(U_{j}). Therefore, there is a point q of int D_{1} which belongs also to 𝔐. Let d_{1} denote the distance between ∂D_{1} and q. Since \{\overline{U_{k}}\}_{k=1,2,···} is a null sequence and 𝔐 is a minimal set, there is an integer L such that

max\{diam f^{L}(\overline{U_{i}}), diam f^{L}(\overline{U_{j}}), d(f^{L}(p_{1}), q)\} ≦ d_{1}/4.

Then, f^{L}(\overline{U_{i}} ∪ \overline{U_{j}}) is contained in int D_{1} just because f^{L}(p_{1}) ∈ f^{L}(\overline{U_{i}}) ∩ f^{L}(\overline{U_{j}}). In particular, f^{L}(C) is contained in int D_{1}. The simple closed curve f^{L}(C) bounds a disc D_{2} contained in int D_{1}. Now the boundary of f^{L−N}(D_{1}) coincides with ∂D_{2}. If f^{L−N}(D_{1}) were different from D_{2}, then our surface would be homeomorphic to the sphere, but this is not the case. Therefore, f^{L−N}(D_{1}) = D_{2} and, in particular, f^{L−N}(D_{1}) ⊂ int D_{1}. Thus f^{L−N}(D_{1} ∩ 𝔐) is contained in D_{1} ∩ 𝔐. By Lemma 1, 𝔐 is not a minimal set of f^{L−N}. In particular, 𝔐 is not a minimal set of f, what contradicts our assumption and shows that \overline{U_{i}} intersects \overline{U_{j}} at most at one point (when i ≠ j).

Next, we assume that there exists a finite chain U_{i_{1}}, U_{i_{2}}, ⋯, U_{i_{n}} (n > 1) such that \overline{U_{i_{j}}} ∩ \overline{U_{i_{j+1}}} ≠ ∅ (j = 1, 2, ⋯, n − 1) and \overline{U_{i_{1}}} ∩ \overline{U_{i_{n}}} ≠ ∅. Then, joining suitable arcs properly embedded in \overline{U_{i_{j}}}, we obtain a simple closed curve C contained in ⋃_{j=1,2,··· ,n} \overline{U_{i_{j}}} (Figure 7). By the same argument as above, this is impossible. This yields our condition (2)

Figure 7:

Finally we will show that the intersection of \overline{U_{i}} and \overline{U_{j}} consists of a locally separating point (if non-empty). Let z denote the unique point of \overline{U_{i}} ∩ \overline{U_{j}}. Choose an arc α contained in \overline{U_{i}} ∪ \overline{U_{j}} and such that α ∩ (∂U_{i} ∪ ∂U_{j}) = \{z\}, α ∩ U_{i} ≠ ∅ and α ∩ U_{j} ≠ ∅. Let V be a neighbourhood of z in T^{2} which is cut by α into two pieces. Then the pathwise connected component W of 𝔐 ∩ V containing z is a neighbourhood of z such that W ∖ {z} is not connected. Thus, z is locally separating. ■

Proof of Theorem 2. First we will show that 𝔐 is connected. Assume that this is not the case. Let 𝔑 be a connected component of 𝔐. By arguments of the proof of Theorem 1, there is N > 1 such that f^{N}(𝔑) = 𝔑 and f^{i}(𝔑) ≠ 𝔑 (i = 1, 2, ⋯, N − 1), and furthermore, 𝔑 is a minimal set of f^{N}. Let g = f^{N}. Denote by V the connected component of T^{2} ∖ 𝔑 containing f(𝔑). Since f^{N+1}(𝔑) = f(𝔑), both, g(V) and V, contain f(𝔑). But, g(V) is also a connected component of T^{2}∖𝔑.

Thus g(V) = V and hence g(∂V) = ∂V. Therefore, ∂V is a g-invariant closed set contained in 𝔑. Since 𝔑 is a minimal set of g, ∂V coincides with 𝔑. By Lemma 9, ∂V is a simple closed curve. Since (as in proof of Theorem 1) 𝔐 has finite number of connected components, this contradicts our assumptions on the structure of our minimal set 𝔐. Therefore, 𝔐 is connected indeed.

By Lemmas 7, 9 and 10, the conditions (1), (2), (3) and (4) of Theorem 2 are satisfied. The remaining problem is the uniqueness of minimal sets. Suppose that there exists another minimal set 𝔐′. Then one of its connected components has to be contained in some connected component U of the complement of 𝔐. By Lemma 6, all the sets of {f^{n}(U)}_{n∈ℤ} are mutually disjoint. Then the orbit starting from a point of the intersection 𝔐′ ∩ U never approaches to this point again, a contradiction. ■

Remark 5. A connected component of ⋃_{i=1}^{∞} \overline{U_{i}} can be invariant under f. Moreover, it is possible that the union ⋃_{i=1}^{∞} \overline{U_{i}} is just connected. Such an example was communicated to the authors by Takashi Tsuboi: Let g be a minimal translation of T^{2}. We choose a point x and a (straight) segment l joining x and g(x). Inserting mutually disjoint discs along the orbit {g^{n}(x)}_{n∈ℤ} as in §1, we obtain a homeomorphism h of T^{2} whose minimal set 𝔐 is homeomorphic to the Sierpiński T^{2}-set. Then the arcs corresponding to {g^{n}(l)}_{n∈ℤ} are contained in 𝔐 and are mutually disjoint. Thus the decomposition with respect to these arcs is shrinkable. By the same argument as in the proof of Theorem 3, we can collapse these arcs and obtain a homeomorphism f such that ⋃_{i=1}^{∞} \overline{U_{i}} is connected and invariant (where, as before, U_{i}’s are connected components of the complement of the minimal set).

Acknowledgement. The results presented here were obtained during the visit of the second author to Uniwersytet Łódzki. He wishes to thank the colleagues of Katedra Geometrii for their hospitality. Also, the third author would like to thank the faculty and stuff of Institut des Mathématiques de Bourgogne (Dijon), where he stayed during the time of preparation of the final version of this paper.

References

[1] J. M. Aarts and L. G. Oversteegen, The dynamics of the Sierpiński curve, Proc. Amer. Math. Soc. 120 (1994) 965–968.

[2] A. Biś, H. Nakayama and P. Walczak, Modelling minimal foliated spaces with positive entropy, preprint.

[3] H. Chu, Algebraic topology criteriafor minimal sets, Proc. Amer. Math. Soc. 13 (1962), 503–508.

[4] R. J. Daverman, Decompositions of manifolds, Pure and Applied Mathematics 124, Academic Press, New York, 1986.

[5] A. Fathi and M. Herman, Existence de difféomorphismes minimaux, Asterisque 49 (1977), 37–59.

[6] E. E. Floyd, A nonhomogeneous minimal set, Bull. Amer. Math. Soc. 55 (1949), 957–960.

[7] W. H. Gottschalk and G. A. Hedlund, Topological dynamics, Amer. Math. Soc. Colloq. Publ. 36 (1955), Providence.

[8] M. Handel, A pathological area preserving C^{∞} diffeomorphisms of the plane, Proc. Amer. Math. Soc. 86 (1982), 163–168.

[9] H. Kato, The nonexistence of expansive homeomorphisms of Peano continua in the plane, Topology Appl. 34 (1990), 161–165.

[10] A. Katok and B. Hasselblatt, Introduction to the modern theory of dynamical systems, Encyclopedia of Mathematics and its Applications 54, Cambridge University Press, Cambridge, 1995.

[11] S. Kim, On low dimensional minimal sets, Pacific Math. J. 43 (1972), 171–174.

[12] K. Kuratowski, Topology II, Academic Press, New York-London; Państwowe Wydawnictwo Naukowe, Warsaw, 1968.

[13] P. D. McSwiggen, Diffeomorphisms of the torus with wandering domains, Proc. Amer. Math. Soc. 117 (1993), 1175–1186.

[14] A. N. Starkov, Minimal sets of homogeneous flows, Erg. Th. and Dynam. Sys., 15 (1995), 361–377.

[15] G. T. Whyburn, Topological characterization of the Sierpiński curve, Fund. Math. 45 (1958), 320–324.

Andrzej Biś Hiromichi Nakayama E-mail: andbis@math.uni.lodz.pl nakayama@mis.hiroshima-u.ac.jp Paweł Walczak Faculty of Integrated Arts and Sciences E-mail: pawelwal@math.uni.lodz.pl Hiroshima University Faculty of Mathematics 1-7-1 Kagamiyama Łódź University Higashi-Hiroshima, 739-8521, Japan Banacha 22 90-238 Łódź, Poland