I have come across this term on several occasions. This time I saw it while reading the papers tsne- a visualization approach in high dimension. http://homepage.tudelft.nl/19j49/t-SNE.html

The way I understood manifolds is through an example in the lecture http://www.iis.ee.ic.ac.uk/~tkkim/mlcv/lecture11_manifold.pdf

Thus the points in high-dimension inherently lie on a low-dimensional manifold.

A manifold can be viewed as a general curve for instance- if points lie on a line – > hyperplane -> any curve. All these can be called by a single name manifold. As per wiki manifold is a subset of Eucledian space which is locally the graph of a smooth function.

There are many papers by the name of manifold learning, so it seems to be a generalization of a hyperplane (linear) or something. [That is what I infer].

Some pointers

1. PCA is a dimensionality reduction techniques that embeds points in a linear space (lower dimension)

2. MDS is also a linear technique.

3. Techniques like isomap are non-linear in that they embed points in space that is non-linear. What do I mean by non-linear space is that the datapoints no-longer satisfy linear properties and measure of distance changes.

Non-linear Space: Two simple interpretetion

1. A simplest example is for each pt in Eucledian space, you construct a new spacewhere you use their squares.

2. The space is curved.