Matrimony and Statistical Tests: An Analogy ?
Caveat: This is not intended to be a reflection of my thoughts, even less of my experience--to date almost non-existent--of this matter and is a purely imaginary account of a fictitious protagonist with a (to the jaundiced eye of the author, at least) realistic setting!
I have been intrigued for quite some time by the analogy between the South Indian (and in general, middle-class India's) matrimonial habits (some would say, tongue-in-cheek, travails) and Statistical Hypothesis testing. Now is the moment for those who know me to pause, purse their lips, and wonder if I have finally come to the a state, signs of which they, ever alert, must have perceived from some time past, viz that I am "off my rocker", "gone bonkers", "over the edge" poor chap, to use evocative language. Those who do not know me must then be expected to look askance and ask themselves, is there probably such a thing as too much of study ? or if a combination of loneliness and excessive academic focus (those who know me too well to swallow this bit about "academic focus" must please humour me on this point at least; I claim Poetic licence!) is indeed such a healthy thing for a young-ish male, after all ?
However, I appeal to the both to judge me as they see fit but only after the conclusion of this rather one-sided view of matrimonial affairs. Onward, therefore, brave reader!
To begin with, let me begin with a suitable Statistician, our Protagonist, the young-ish male, South Indian bram, who is about to choose a prospective mate; let further this agent be that quintessential creature, one who is modern yet traditional, one who attempts to bridge the (hopefully rather small) chasm between the both worlds, and may (or may not) posses that most important of characteristics, one whose presence enables fond prospective in-laws to smile indulgently at his other 'drawbacks' and to tolerate other 'whims' which, in those devoid of this character, would be labelled uppityness or downright bad habits, the Software Job, that ultimate Test of a Man's Worth.
I will now proceed to analyse the options faced by this agent, in some detail. We being with this Young-ish man, one fine evening, after successfully resisting pressures, both internal and external, to find a 'suitable mate', realising that his position, from a prospective in-law's view, is very similar to that of a stock which is at its peak, and if the encashing is not done rather soon, there maybe a crash in the value (for those more poetically inclined,“There is a tide in the affairs of men, Which taken at the flood, leads on to fortune. Omitted, all the voyage of their life is bound in shallows and in miseries. On such a full sea are we now afloat. And we must take the current when it serves, or lose our ventures.”). So reasoning, our agent decides to undertake to remedy this deficiency of his, ("Such is the folly of youth", quoth Malignant Fate, standing by) and immediately comes up against the quintessential problem in these types of marriages: specification of the parameters of a partner.
He is usual in this, having always vaguely thought about a good looking, tall-ish, fair, well (but not-too-well) educated , traditional but not too much, (recalls a particular specimen with some degree of fright), modern-and then pauses with his pulse quickening with thoughts of the too-modern-type-of-female fresh in his mind, resolves that she ought not to be too modern, type of girl. Naturally, he expects her to be from his own caste/sub-caste, speak the same language(s), and hopes she is either in the same city (or is willing to shift, naturally). He then recalls that he wishes his wife to work, in a not too taxing type of job but of course, she ought not to be too pushy or ambitious (so unseemly, really).
It is here that he fully realises his dependence on his mother, since only the females of elderly aspect seem even part capable of arranging such things. Like the impatient chap that he is, he directly approaches his mother on the issue, who is alike surprised at both his delayed response and his hurry, now that he has responded. He then gets onto his mother's nerves, with his constant questioning about whether she has found a "suitable" girl, to which his mother finally replies, testily, if he expects her to pull a girl, to his exact specifications, out of a hat ? This free and frank (on the part of his mother, at least) exchange naturally leaves him dissatisfied, yet he keeps a lower profile over the next fortnight, knowing well his dependence on his mother for her rather extensive contacts, his father already being dismissed as being incapable of so important a task as finding a suitable mate for him. He then waits with growing impatience, and feels that in more than a month, he has barely seen a couple of photographs, and heard of maybe 3-4, being used to scales of mere weeks on most other tasks.
This is when he is reminded of his knowledge of Statistical Hypothesis Testing (SHT), not because it is very relevant to the case on hand, but because it is what he is intellectually interested in, at the moment (a far stretch, I agree, to assume that Software Types know of/use SHT--Emerson, I cry--See Note 1). He vaguely recalls that all tests begin with having a sample or realisation from which one must infer some aspect of the population, using a metric called an estimator. Further, it flashes on him that (in the parametric test at least!) the statistic can be made to have "good" properties.
Intrigued, he then purususes the thing to its logical conclusion, and reflects that, just as in the SHT case, he has merely a one realisation (sample), from which he must infer a population characteristic (behaviour of the proposed mate); he has "estimators" of this characteristic, in metrics for educational achievement, Job type and salary, photograph (associated characteristics) on which he might perform "tests"; however, here he has to ponder more deeply, since he is forced to the conclusion that, unlike in the SHT case, where there is "a' metric, here has a cornucopia of them; hence, he not only has to worry about the "distribution" of the statistic but also question the meaning of one ?
He then realises the reasons for his increasing disquiet, which is that in Statistical tests, there are "Type I" and "Type II" errors, and a maximum power, with not both being minimised simultaneously! Thus, he reasons, even if he picks what he thinks is the girl with the "correct" specification, he nonetheless has a non-zero chance of making the "incorrect" choice, and that he has an equally good chance of rejecting the "better" girl! Worse, his "tests" generally have low(or no) power against locally superior alternatives, such as girl better-placed but similar looking. Finally, he realises that all his testing might be no good anyway since the best he can do is to run competing tests and "reject" the null, but if he has rejected two non-nested nulls, he now has to devise a test for these non-nested models, and of these Theory provides him with very little help (at least, that he can understand, since Statisticians are hardly the most lucid of writers!)
He then feels that he does not even have the satisfaction of taking recourse to that dead horse of Statistics, the bootstrap, since his mother would most definitely look askance at such an idea; after all, which mother likes to be told to be ready to supply photographs of an innumerable number of "suitable" girls?
In fact, this so occupies the moments of our friend that he so far forgets himself as to stop bothering his mother regarding "suitable Girls", something which he is immediately recalled to with his mother deluging him with photographs and information. Recalled, but not yet reconclied, to the real-world, he begins by cursing Statistics for being so useless a field, but then realises eventually that if even three-quarters-of-a-century of Statistics is unable to solve his problems, they must indeed be egregious, and this reflection brings to our friend's soul immense peace, since if it matters are so far out of his hands, he is not responsible for his fate (he still has strange ideas regarding control of his fate; marriage, naturally, will disabuse him soon of all such quaint notions).
In fact, this reflection lightens his mood so much as to render his task of sifting through these photographs less of a burden than before and each CV he scrutinises merely cursorily. The reason is thus: he is finally reminded of the advice one of his faculty gave him on testing: never carry out point testing except to confirm your hypothesis, most of them are either incorrect or pointless, and in the end, whether the hypothesis is rejected (or is not) has no bearing on the question being asked. He thus begins to trust in luck as much as in judgement, and to go in hope, with the following on his lips "..all is for the best, in this the best of all possible worlds"!
Thus was Mated a Statistician with his Master..oops Wife!
I have been intrigued for quite some time by the analogy between the South Indian (and in general, middle-class India's) matrimonial habits (some would say, tongue-in-cheek, travails) and Statistical Hypothesis testing. Now is the moment for those who know me to pause, purse their lips, and wonder if I have finally come to the a state, signs of which they, ever alert, must have perceived from some time past, viz that I am "off my rocker", "gone bonkers", "over the edge" poor chap, to use evocative language. Those who do not know me must then be expected to look askance and ask themselves, is there probably such a thing as too much of study ? or if a combination of loneliness and excessive academic focus (those who know me too well to swallow this bit about "academic focus" must please humour me on this point at least; I claim Poetic licence!) is indeed such a healthy thing for a young-ish male, after all ?
However, I appeal to the both to judge me as they see fit but only after the conclusion of this rather one-sided view of matrimonial affairs. Onward, therefore, brave reader!
To begin with, let me begin with a suitable Statistician, our Protagonist, the young-ish male, South Indian bram, who is about to choose a prospective mate; let further this agent be that quintessential creature, one who is modern yet traditional, one who attempts to bridge the (hopefully rather small) chasm between the both worlds, and may (or may not) posses that most important of characteristics, one whose presence enables fond prospective in-laws to smile indulgently at his other 'drawbacks' and to tolerate other 'whims' which, in those devoid of this character, would be labelled uppityness or downright bad habits, the Software Job, that ultimate Test of a Man's Worth.
I will now proceed to analyse the options faced by this agent, in some detail. We being with this Young-ish man, one fine evening, after successfully resisting pressures, both internal and external, to find a 'suitable mate', realising that his position, from a prospective in-law's view, is very similar to that of a stock which is at its peak, and if the encashing is not done rather soon, there maybe a crash in the value (for those more poetically inclined,“There is a tide in the affairs of men, Which taken at the flood, leads on to fortune. Omitted, all the voyage of their life is bound in shallows and in miseries. On such a full sea are we now afloat. And we must take the current when it serves, or lose our ventures.”). So reasoning, our agent decides to undertake to remedy this deficiency of his, ("Such is the folly of youth", quoth Malignant Fate, standing by) and immediately comes up against the quintessential problem in these types of marriages: specification of the parameters of a partner.
He is usual in this, having always vaguely thought about a good looking, tall-ish, fair, well (but not-too-well) educated , traditional but not too much, (recalls a particular specimen with some degree of fright), modern-and then pauses with his pulse quickening with thoughts of the too-modern-type-of-female fresh in his mind, resolves that she ought not to be too modern, type of girl. Naturally, he expects her to be from his own caste/sub-caste, speak the same language(s), and hopes she is either in the same city (or is willing to shift, naturally). He then recalls that he wishes his wife to work, in a not too taxing type of job but of course, she ought not to be too pushy or ambitious (so unseemly, really).
It is here that he fully realises his dependence on his mother, since only the females of elderly aspect seem even part capable of arranging such things. Like the impatient chap that he is, he directly approaches his mother on the issue, who is alike surprised at both his delayed response and his hurry, now that he has responded. He then gets onto his mother's nerves, with his constant questioning about whether she has found a "suitable" girl, to which his mother finally replies, testily, if he expects her to pull a girl, to his exact specifications, out of a hat ? This free and frank (on the part of his mother, at least) exchange naturally leaves him dissatisfied, yet he keeps a lower profile over the next fortnight, knowing well his dependence on his mother for her rather extensive contacts, his father already being dismissed as being incapable of so important a task as finding a suitable mate for him. He then waits with growing impatience, and feels that in more than a month, he has barely seen a couple of photographs, and heard of maybe 3-4, being used to scales of mere weeks on most other tasks.
This is when he is reminded of his knowledge of Statistical Hypothesis Testing (SHT), not because it is very relevant to the case on hand, but because it is what he is intellectually interested in, at the moment (a far stretch, I agree, to assume that Software Types know of/use SHT--Emerson, I cry--See Note 1). He vaguely recalls that all tests begin with having a sample or realisation from which one must infer some aspect of the population, using a metric called an estimator. Further, it flashes on him that (in the parametric test at least!) the statistic can be made to have "good" properties.
Intrigued, he then purususes the thing to its logical conclusion, and reflects that, just as in the SHT case, he has merely a one realisation (sample), from which he must infer a population characteristic (behaviour of the proposed mate); he has "estimators" of this characteristic, in metrics for educational achievement, Job type and salary, photograph (associated characteristics) on which he might perform "tests"; however, here he has to ponder more deeply, since he is forced to the conclusion that, unlike in the SHT case, where there is "a' metric, here has a cornucopia of them; hence, he not only has to worry about the "distribution" of the statistic but also question the meaning of one ?
He then realises the reasons for his increasing disquiet, which is that in Statistical tests, there are "Type I" and "Type II" errors, and a maximum power, with not both being minimised simultaneously! Thus, he reasons, even if he picks what he thinks is the girl with the "correct" specification, he nonetheless has a non-zero chance of making the "incorrect" choice, and that he has an equally good chance of rejecting the "better" girl! Worse, his "tests" generally have low(or no) power against locally superior alternatives, such as girl better-placed but similar looking. Finally, he realises that all his testing might be no good anyway since the best he can do is to run competing tests and "reject" the null, but if he has rejected two non-nested nulls, he now has to devise a test for these non-nested models, and of these Theory provides him with very little help (at least, that he can understand, since Statisticians are hardly the most lucid of writers!)
He then feels that he does not even have the satisfaction of taking recourse to that dead horse of Statistics, the bootstrap, since his mother would most definitely look askance at such an idea; after all, which mother likes to be told to be ready to supply photographs of an innumerable number of "suitable" girls?
In fact, this so occupies the moments of our friend that he so far forgets himself as to stop bothering his mother regarding "suitable Girls", something which he is immediately recalled to with his mother deluging him with photographs and information. Recalled, but not yet reconclied, to the real-world, he begins by cursing Statistics for being so useless a field, but then realises eventually that if even three-quarters-of-a-century of Statistics is unable to solve his problems, they must indeed be egregious, and this reflection brings to our friend's soul immense peace, since if it matters are so far out of his hands, he is not responsible for his fate (he still has strange ideas regarding control of his fate; marriage, naturally, will disabuse him soon of all such quaint notions).
In fact, this reflection lightens his mood so much as to render his task of sifting through these photographs less of a burden than before and each CV he scrutinises merely cursorily. The reason is thus: he is finally reminded of the advice one of his faculty gave him on testing: never carry out point testing except to confirm your hypothesis, most of them are either incorrect or pointless, and in the end, whether the hypothesis is rejected (or is not) has no bearing on the question being asked. He thus begins to trust in luck as much as in judgement, and to go in hope, with the following on his lips "..all is for the best, in this the best of all possible worlds"!
Thus was Mated a Statistician with his Master..oops Wife!
Note 1: The Cry "Emerson" was first used, to my knowledge, by Asimov in one of his text books-with co-authors--on bio-chemistry!! The Cry refers to Asimov's reference to Emerson's statement: "A Foolish consistency is the hobgoblin of little Minds" to smooth out certain inconsistencies, and is taken as a cry descrying expectations of perfect consistency in prose!!
posted by Unknown at
Tuesday, April 03, 2007
|
1 Comments
