Continuous Hierarchy?

Suppose I came to you and said, "I want a mathematical approach for working with with data in a continuous multidimensional space". Hopefully you'd point me at linear algebra.

Suppose I came to you and said, "I want a mathematical approach for working with data that's connected via arbitrary relationships". Hopefully you'd point me at predicate logic, maybe graph theory, or maybe even category theory (I only vaguely understand category theory, so I may be wrong there).

So now I come to you and say, "I want a mathematical approach for working with data that's connected by arbitrary continuous relationships". What theory would you point me at?

By continuous, I mean that it's possible to smoothly interpolate between any two relations. This necessarily requires that relations themselves aren't strictly binary. It should also be possible to build either hierarchical or meta relationships, ala relational calculus or higher order predicate logic. Ideally, relationships would themselves be objects (i.e. the theory should be impredicative).

Above all, the theory should be quantitative rather than symbolic (I believe that mostly rules out category theory, or any theory based on logic). It doesn't necessarily have to be computationally tractable, though.

Is there a standard mathematical framework for this kind of thing? I think I'd recognize it if I saw it through the right lens, but I have the feeling that my qualitative descriptions of what the theory should look like are bordering on meaningless.

I've been obsessed with this idea for the past few weeks. It would be nice if someone had already done the hard work of exploring it.

Posted on November 20, 2006 01:59 PM
More projects articles

Comments

Probability theory?

Posted by: dave glasser at November 20, 2006 03:40 PM

How can I represent hierarchy with probability distributions?

Posted by: Kim at November 20, 2006 07:41 PM

Category theory is actually one level above linear algebra, logic, graph theory and so on. You can think of it as abstracting away the differences between them, so you can talk about mathematical concepts independently of any concrete theory that implements them. (In that sense, I think of category theory as the mathematical equivalent of design patterns.)

Whatever this theory is, you'll be able to study it in the language of category theory.

Posted by: Pseudonym at November 20, 2006 09:21 PM

Could you give a concrete example of smoothly interpolating between two relations? Sounds interesting...

Posted by: Chris at November 21, 2006 01:01 AM

Chris, that's the problem, I'm not sure what I'm talking about here.

I'm somewhat reluctant to describe my real motivation, because I'm afraid it will bias the answer, but here's a clue. I want to start with a defined universe of data points. Then I want to read in a sequence of relationships between those data points. I want to "summarize" those relationships in order to produce a final relationship whose representation is sublinear with respect to the size of the input (constant would be best, but logarithmic would probably be okay). An analogy to what I'm thinking would be if data points were multidimensional coordinates, and relationships were matrices. Then I could sum up relationships just by multiplying matrices -- the size of the final matrix would then be independent of the number of original matrices. The only problem is that matrices are too simple -- they can't represent structure at all. I also need the original relationships to be somewhat like probability distributions, in that they are capable of representing continuous shades of grey. Dave was on the right track by suggesting probability theory, but I don't know how to make probability distributions represent hierarchy/structure/meta.

So here's one idea that's somewhat more concrete. You're given a set of data points, each of which is defined as a probability distribution over all the other data points (and maybe even over itself as well). Now you can augment the domain by introducing even more probability distributions (new data points), which range over the original data points plus all the newly introduced distributions. Modify the original data points so that they too can range over these newly introduced points. Take that idea to the limit, so that you have an infinite number of probability distributions, all of which range over each other. Now you have some way of talking about continuous relations (they're just probability mass functions), and there's some structure (since each point is itself a pmf).

Here's a different idea. Maybe what I want is to represent data points as coordinates in a non-cartesian space. Maybe I'm given a set of quantitative relationships between points, and what I want to do is to discover the structure of the space that those relationships describe. This might be a variation on topology. Or it might be some version of principal component analysis for warped space.

What I want is a mathematical theory that does something vaguely like this in a principled way. What I especially want is to find out that there are a few decades of work behind the theory already, so that I can reuse its insights.

Posted by: Kim at November 21, 2006 05:49 AM

This makes me think of neural nets. You can represent relations as a layer of nodes with weighted links from the data points. The value of each data point is multiplied by the weight of a link; the links are summed for each node, and a "sigmoid function" is applied, mapping (-inf->inf) to [-1,+1], representing a classification, but in a smooth continuous way. A second layer of nodes can then take inputs from the first layer, or you can mix the first and second layers if you don't mind things getting complicated.

At this point your representation is at least N^2; definitely not sublinear. But maybe depending on your application, you can consider your "summary" representation to be made up of only links to higher-order nodes (i.e. only inputs from other nodes).

A neural network layer can be construed as a matrix multiplication; if your data points are scalars, and the collection of them is a vector, then each node influenced by them is a dot product with a set of link weights; and a group of those nodes is a matrix multiplication resulting in a new vector (representing those nodes' levels of activation). Two layers have more computational/classificational power than a simple matrix multiplication because of that sigmoid function applied in between. Without that, two layers could be reduced to one with no loss in expressive power.

Neural nets feel more like engineering than math to me, but your discussion of matrices kind of brought it to mind, so I thought I'd mention it in case it sparks some useful insight for you...

Posted by: Chris at November 21, 2006 09:46 AM

Differential geometry.

Posted by: Tobin Fricke at November 21, 2006 06:43 PM

You may want to check out network of network models, though I think you'd become quickly dissatisfied. Neural nets are just a special view of linear algebra and probability.

It sounds like you're looking for something that combines analysis (eg, where probability theory comes from) and algebra (the study of simple relationships).

One of my roomates from last year studies something called representation theory. Algebraic/combinatoric topology would be easier to handle, but sounds further removed.

Posted by: leo at November 22, 2006 11:23 AM

When I described this to Micha Elsner at Brown, he pointed me at Markov Logic Networks. I think that may be the closest I've seen to what I'm looking for, but it still assumes that the set of predicates and their truth values are known a priori. I'm looking for something that would let me derive a maximum likelihood estimate of a relational model, where the relations are hidden parameters.

Posted by: Kim at November 28, 2006 09:59 PM

How about fuzzy logic and fuzzy systems? Basically, you need to hack set theory into something continuous. Another thing you should look at is temporal logic, multiple futures.

Posted by: John Carlson at April 4, 2008 05:45 PM

Did you figure out a solution? I'd be interested in knowing more about it.

Posted by: S at April 27, 2008 07:14 PM

Consider "Real Recursive Functions" and "Differential Recursion"

Posted by: John Carlson at August 2, 2009 02:22 AM

Consider "Differential Propositional Calculus"

Posted by: John Carlson at August 26, 2009 12:31 PM

ugg boot australia
bailey button ugg sale
ugg classic cardy cheap
classic cardy ugg sale online
ugg unique bailey button

ugg bailey button unique

Posted by: ugg boots on sale and free shipping at October 11, 2009 12:23 AM

Thanks for your useful info, I think it’s a good topic. So would you like the info about the
jordan shoes
Air jordan shoes
cardy ugg boots
Tiffany Jewelry
ugg bailey button boots
cheap jordan shoes
ugg boots uk
Michael jordan shoes
ugg london
ugg nightfall boots
ugg boots sale
classic ugg boots
sundance ugg boots
air jordan fusion 6
air jordan 6 rings
jordan shoes for sale

Posted by: xinxi at October 23, 2009 10:52 PM

Good ! I like it ! Welcome to read following news: ugg boots,Nike dunk sb,discount ugg boots, uggs ,nike dunk low,ugg boots sale,UGG Bailey Button boots,new nike sb,UGG Classic short boots,UGG Nightfall boots,cheap nike dunks,UGG Classic tall boots.

Posted by: ugg boots at October 27, 2009 10:47 PM

Tiffany & Co Man In The Moon Pendant Necklace [N019] On Sale
Tiffany & Co Heart Link Necklace [N015] On Sale
Tiffany & Co Heart Necklace N185 [N185] On Sale
Tiffany & Co Heart Pendant Necklace N209 [N209] On Sale

Tiffany & Co Round Pendant Necklace N219 [N219] On Sale
Tiffany & Co Frank Gehry Torque Pendant Necklace [N201] On Sale

Posted by: vuitton at October 28, 2009 08:27 AM

Tiffany & Co Round Pendant Necklace N219 [N219] On Sale
Tiffany & Co Frank Gehry Torque Pendant Necklace [N201] On Sale

Posted by: vuitton at October 28, 2009 08:28 AM

Tiffany & Co Elsa Peretti 5 Beans Bracelet [S004] On Sale
Tiffany Bracelets On Sale,Discount Tiffany & Co. Silver Bracelets
Tiffany & Co Continuous Teardrop Bracelet [S029] On Sale
Tiffany & Co Double Venetian Link Bracelet [S033] On Sale

Posted by: discount tiffany jewelry at October 29, 2009 06:31 AM

Posted by: discount tiffany jewelry at October 29, 2009 06:32 AM

Love this man. Great stuff, great prices, free shipping. Email him if you cant find what you are looking for and he will look for you. Good customer service.louis vuittonSomeone who is knowledgeable in louis vuitton.Another guy from hong Kong (Largest louis vuitton second hand city, I believe) he has great stuff at fair prices but his shipping is insane

Posted by: louis vuitton at October 29, 2009 10:19 AM

Posted by: louis vuitton at October 29, 2009 10:20 AM

Supra Shoes
Mens Supra Cruizer Shoes ---Special
Men's Supra Cruizer Red Shoes
Mens Supra Cruizer Cool Shoes
Mens Supra Cruizer Shoes --Charming
Mens Supra Cruizer Pure black Shoes
Mens Supra Cruizer Shoes --Leisure
Man's Supra Cruizer Shoes
Men's Supra Muska Skytop Shoes
Men's Supra Skateboarding Shoes
Men's Supra Strapped Shoes
Men's Supra Suprano High Shoes
Women's Supra Muska Skytop Shoes
cheap ugg boots sale
uggs classic mini sale
UGG Ultra Tall Boots
Cheap Ugg Boots
UGG Classic Tall Boots
UGG Classic Cardy
UGG Classic Short Boots

Posted by: supra shoes at November 1, 2009 12:50 PM

Good day! Thx for your great post and Im thinking about how to introduce my ugg boots to you cos Im not sure if u like this. Many people who live in the cold area like the ugg boots, especially the north-europe. Our uggshare.co.uk supply the cheapest classic cardy ugg boots, class tall ugg boots, classic short ugg boots and other series of uggs, hope my words didn’t trouble u, cy!

Posted by: ugg boots at November 16, 2009 09:37 PM

discount ugg boots
forture
ugg boots
Discount store

Posted by: wholesaler business website, net shopping site, external wholesaler,dvd wholesaler,wholesaler definition,china wholesaler,mutual fund wholesaler,define wholesaler,insurance wholesaler at November 17, 2009 04:08 AM

Rolex Sea-Dweller watch for sale
Rolex Day Date watch for sale
Rolex Masterpiece watch for sale
Rolex Milgauss watch for sale

Posted by: rolex replica at November 19, 2009 12:38 AM

What is good for winter in 2009? I think the new UGG Mayfaire Boots
5116 is a good choice for lady.And the UGG Classic Cardy Boots makes you special.If you want to be warm in the cold days,the UGG Classic Tall Boots makes your dream come true.And aslo the UGG Women's Bailey Button Boot is as fashion as comfortable.There is a special ugg boots for you.

Posted by: uggs on sale at November 19, 2009 03:59 AM

discount nike air yeezy

wholesale online"> china wholesale online

discount ugg

boots
nike free shoes
furture
ugg boots
Discount store

Posted by: nike-airyeezy at November 19, 2009 09:44 PM

We are the best online sales for the china wholesale . Here you can have a large of choices of kinds Ugg Boots,Converse Shoes,Timberland Boots,puma shoes,Nike Shox Shoes ,Nike Dunk SB Shoes,Nike Air Max，Links Of London,Tiffany Jewelry,Dior Handbags?,jimmy choo handbags ,Cartier Watches, 8GB Mp4 Players,Bluetooth Car DVDs. All our cheap online cheap goods are high quality and original packages, and best service. We offer our customers the best service, 7 days arrive at your door.Enjoy your easy and happy shopping with us.

Posted by: cheap goods sale. at November 20, 2009 01:09 AM

Rolex replica
Rolex Air-King replica
Rolex Datejust II replica
Rolex Submariner replica
Yachtmaster replica Montblanc watches replica
movado watches replica
omega watches replica
oris watches replica
Panerai watches replica

Posted by: Rolex GMT watches at November 20, 2009 02:29 AM

Burberry watches
Bvlgari watches
Cartier watches
chopard watches
Concord watches
croum watches
DeWitt watches
ebel watches
Ferrari watches

Posted by: Bell & Ross watches at November 20, 2009 08:06 PM

Cartier watch for sale
chopard watch for sale
croum watch for sale
DeWitt watch for sale
ebel watch for sale
Ferrari watch for sale
Franck Muller watch for sale
Glashutte watch for sale
Graham watch for sale
Hermes watch for sale

Posted by: Cartier watch for sale at November 20, 2009 08:09 PM

U-boat watches
Vacheron Constantin watches
Rolex watches
Rolex Air-King watches
Rolex Datejust II watches

Posted by: Lady Watch watches at November 20, 2009 08:22 PM

Tiffany Necklaces
Tiffany sets
Tiffany Accessories
replica Tiffany Earrings
replica Tiffany Necklaces
replica Tiffany sets
replica Tiffany Accessories

Posted by: fashion Tiffany Earrings at November 20, 2009 08:27 PM

replica Glashutte
replica Graham
replica Hermes
replica Hublot
replica IWC

Posted by: replica Burberry at November 21, 2009 03:20 AM

replica Glashutte
replica Graham

Posted by: replica Burberry at November 21, 2009 03:20 AM

movado watch for sale
omega watch for sale
oris watch for sale

Panerai watch for sale
Patek Philippe watch for sale
Piaget watch for sale

Posted by: Rolex GMT watches at November 22, 2009 11:58 AM

movado watch for sale
omega watch for sale
oris watch for sale

Panerai watch for sale
Patek Philippe watch for sale
Piaget watch for sale

Posted by: Rolex GMT watches at November 22, 2009 11:59 AM

movado watch for sale
omega watch for sale
oris watch for sale

Panerai watch for sale
Patek Philippe watch for sale
Piaget watch for sale

Posted by: Rolex GMT watches at November 22, 2009 11:59 AM

movado watch for sale
omega watch for sale
oris watch for sale

Panerai watch for sale
Patek Philippe watch for sale
Piaget watch for sale

Posted by: Rolex GMT watches at November 22, 2009 11:59 AM

Your article is write very well, I like it very much ~
ghd Hair Straightener
ghd Hair Straighteners
ghd straighteners
GHD
hair straighteners
bose in ear headphones
bose on ear headphones
bose headphones
nike sb dunks
nike dunks
I wish you have a wonderful day!Thank you.

Posted by: ghd straighteners at November 22, 2009 08:36 PM

Supra Shoes
Supra Skytop Shoes
Supra Society
Supra Thunder
Supra Suprano Hi
Supra Strapped
Supra Trinity
Supra Skylow 1.5
Supra Dixon
Diablo 1.5
Supra Cruizer
Supra Vaider Shoes
Supra Cuban

Posted by: Supra Shoes at November 22, 2009 11:15 PM

coach bags cheap
nike air force 1 low premium
timberland boots for sale
cheap ugg boots sale
wholesale clothing
mbt shoes clearance

Posted by: discount ugg boots at November 23, 2009 12:42 AM

discount Christian Louboutin Pump
discount Christian Louboutin sandal
discount Jimmy Choo Handbags
discount Jimmy Choo Shoes
discount Yves Saint Laurent Shoes

Christian Louboutin Boot on sale
Christian Louboutin Pump on sale
Christian Louboutin sandal on sale
Jimmy Choo Handbags on sale

Posted by: Jimmy Choo Handbags on sale at November 23, 2009 01:04 PM

discount Christian Louboutin Pump
discount Christian Louboutin sandal
discount Jimmy Choo Handbags
discount Jimmy Choo Shoes
discount Yves Saint Laurent Shoes

Christian Louboutin Boot on sale
Christian Louboutin Pump on sale
Christian Louboutin sandal on sale
Jimmy Choo Handbags on sale

Posted by: Jimmy Choo Handbags on sale at November 23, 2009 01:09 PM

cheap Christian Louboutin Boot
cheap Christian Louboutin Pump
cheap Christian Louboutin sandal
cheap Jimmy Choo Handbags
cheap Jimmy Choo Shoes
cheap Yves Saint Laurent Shoes

Posted by: Jimmy Choo Handbags at November 23, 2009 02:35 PM

rongTiffany Jewelry Online - Discount Tiffany & Co Jewelry On Sale

Hot Sell Tiffany Jewelry - Tiffany Jewelry Online

Return To Tiffany Double Heart Detachable Pendant Necklace - Tiffany Jewelry Online

Tiffany&Co 1837 Titanium I.D. Bracelet - Tiffany Jewelry Online
Tiffany & Co Bead Bracelet - Tiffany Jewelry Online

Tiffany&Co Atlas Round Pendant Necklace - Tiffany Jewelry Online
Tiffany&Co Bar Pendant Necklace - Tiffany Jewelry Online
Tiffany&Co Bead Necklace(N242) - Tiffany Jewelry Online
Tiffany&Co Bean Pendant Necklace(N157) - Tiffany Jewelry Online

Posted by: at November 24, 2009 11:08 PM

Laptop Battery Laptop Battery Laptop Batteries
Laptop Batteries discount laptop battery
discount laptop battery
notebook battery notebook battery
computer battery computer battery
replacement laptop battery replacement laptop battery
notebook batteries notebook batteries

Posted by: Laptop Battery at November 25, 2009 12:08 AM

People all over the world know the abercrombie and fitch,but not everyone really knows how fashion the abercrombie is,hollister is the Legend maker. Everybody wears the hollister clothing would be the abercrombie mens and the abercrombie womens, if you want know you can search the Ruehl No.925 or abercrombie outlet in the www.google.com .

Posted by: fitch at November 25, 2009 10:06 PM

China Electronics Wholesale China

Phone Cheap Cell Phones

href="http://www.thechinaphone.com/index.php?main_page=advanced_search_result&keyword=China%20Wholesale">China

Wholesale

replica handbags
Louis Vuitton Replica
Louis Vuitton handbags
fake Louis Vuitton bags
fake Louis Vuitton
replica rolex
replica rolex
fake rolex
buy cheap fake rolex
rolex Submariner

Posted by: lovebo at November 29, 2009 08:08 AM

replica watch
ebel replica
replica Glashutte
Rado watch for sale
replica Hublot
lastmiss

Posted by: replica Glashutte at November 30, 2009 02:47 PM

It is no doubt that Replica Louis Vuitton Speedy 30 enjoys highest reputation in the world. Louis Vuitton Speedy 30, purses, shoes, watches, clothes, and belts successfully seize loving heart of a majority of people. Speedy 30 is filled with romantic and fantasy, and it is a symbol of noble and high quality of life for gucci ever! Today, let us talk about Tivoli PM.belts have different styles and colors, such as Monogram series belts, Graffiti series belts, both men and women love them crazily. All of them match with various leisure clothes, formal business suits, and beautiful clothes very well. Whenever you wear a Louis Vuitton belt to make your public appearance, you will be naturally concerned and admired by every body. When we talk of high quality, there is no other company to compare with Louis Vuitton Replica Speedy 30. All Louis Vuitton belts adopt top quality cow leather, high quality hard ware, and the color will never be faded. The belts are so durable, and you can wear them to match your clothes for several years. Also, you can clearly see an obvious LV logo as a buckle of each belt.

Posted by: fanqin at December 1, 2009 02:14 AM

Posted by: Jimmy Choo Handbags at December 1, 2009 04:33 AM

replica watches
replica Rolex Day-Date II
Patek Philippe watches
Hublot watches
Rolex Air-King watch for sale
Tiffany&Co Heart Clasp Link Necklace

Posted by: nike sports at December 1, 2009 02:09 PM

watches replica
Longines watches
Patek Philippe replica
replica Rolex Air King
Panerai watch for sale
breitling swiss replica watches

Posted by: replica Audemars Piguet at December 1, 2009 09:09 PM

ugg boots sale at a time when large heat has set off a winter 08/09 to Ewha.Outlines the

warm wool winter fashion degree to wool crocheted wool ugg boots uk to join after the

body does not stimulate your skin soft, soft comfort UP. Fashion can not without
ugg boots presence in Winter.and ugg boots have many series.such as:
ugg classic
UGG Bailey Button boots
ugg nightfall
ugg kids

Posted by: uggboots at December 2, 2009 08:43 PM