Saturday, May 30, 2015

Fwd: [datameet] Fallen documentary

I remember during my teen years, coming across the historical narratives of world ward 1 and 2, especially the second one, how much more aware it had made me about why all this democracy, liberty and respecting-peoples-choices business is so important, and what happens when we belittle it. Please make sure your child gets to know about this all, in full detail, before he/she reaches adulthood.

The Fallen of World War II is an interactive documentary that examines the human cost of the second World War and the decline in battle deaths in the years since the war. The 15-minute data visualization uses cinematic storytelling techniques to provide viewers with a fresh and dramatic perspective of a pivotal moment in history.

The film follows a linear narration, but it allows viewers to pause during key moments to interact with the charts and dig deeper into the numbers.

-- from datameet group.

Article: Nonviolent Power in Action: An Interview with Dennis Dalton

Dr. Dennis Dalton, one of the world's leading experts on Mahatma Gandhi, recently visited CSRC, meeting with students in the Center's Undergraduate Research Fellows program and in Hardt-Nickachos Peace Studies Chair Yasmin Saikia's HST 394 class, Gandhi and the Politics of Non-violence. Dalton, author of The Indian Idea of Freedom and Mahatma Gandhi: Nonviolent Power in Action, as well as numerous other books and articles, is professor emeritus from Columbia University. During his visit, Dalton also sat down with the Center's communications assistant and former CSRC undergraduate research fellow, Richard Ricketts, for the following interview. Their talk reflects the depth and breadth of Dalton's interests, including classical and modern, Western, and Asian political theory; politics of South Asia, particularly the Indian nationalist movement; nonviolence and violence in society; and ideologies of modern political movements in Europe, India, China, and Africa. The interview was conducted, condensed and edited by Richard Ricketts.

What do you see as the resistance to peace studies as a field?

That is a hard question. You have at many universities, institutes and centers that incorporate the word "peace," such as the Institute of War and Peace studies at Columbia, but all they typically study is the war side. Peace is seen as passive, the absence of war, so it is thought that there is nothing there to study.

I suspect this line of thinking is encouraged, in part, by the large grants that are provided from the Department of Defense. Columbia, for example, has traditionally been funded very liberally by the Defense Department and, in line with my suspicions, the money is not there for peace studies. That is why I was really interested in this program at ASU. I met Ann Hardt yesterday and she was really committed. You need someone like that who can fund these types of programs.

In additional to that, when I went to the instruction/education board meetings [at Columbia], I was told the subject was academically soft, not rigorous enough. What they meant was that they just deemed pacifism as a soft weak effeminate subject.

What do you as some of the most important and influential intellectual/theoretical contributions from Gandhi?

I taught Gandhi in a political theory course for 39 years at Barnard. I struggled for almost as many years to get Columbia to include Gandhi, or King, in their core curriculum. They include Franz Fanon, for example, in their core curriculum, which goes all the way back to Homer. But they would not include Gandhi, not even consider it, because he was considered not to be rigorous or sophisticated enough in his theory, that he hadn't made a profound enough contribution to political thought. There is a very strong bias against him, I think this is so unjust and unfair to Gandhi.

The argument I make is that he develops distinct and original understandings in the context of, at the very least, two ideas, freedom and power. Swaraj, freedom, he argued, contrary to most political theorists in the West, must have both an external and internal quality. One is not free unless one is liberated in an external sense, that is, one must be politically and socially "free" from the British. Additionally, one is not free unless one is also liberated in an internal sense as well, one must be free of anger, of hatred, of all those obsessions and addictions that control us. Those forms of enslavement are with us and they must be expunged in order for a country/people to be truly and completely free. So Gandhi made the statement that India may well get its independence but that will be as nothing if they have not emancipated themselves from the exclusiveness of ideas such as untouchablity, the commitment and belief of the untouchable as an Other, or the prejudice of Hindu/Muslim enmity. Gandhi saw freedom as liberation in the most complete sense.

Secondly, satyagraha is an entirely original concept in the sense he used it. The word 'nonviolence' was originated by Gandhi, you can't find in the vocabulary before he started it 1920's. I think it is unfortunate that in the English language it has to be "non" something, that we don't have a positive statement about nonviolence. The closest we can come to it is compassion, but that doesn't convey the force and power that Gandhi wanted satyagraha to convey. Some historians have argued that he is one of the most original thinkers of our age, and it is especially true in terms of freedom and power. I think the proof is there that his work is academically sophisticated and rigorous.

How does Gandhi's work shed light on the relationship between religion, politics, and violence?

Gandhi said repeatedly from the first page of his autobiography that religion and politics are inextricably interwoven; we cannot understand one without the other. He was sometimes called a charlatan because of that, some felt he was using religion to promote his political causes, and that can be charged to any number of religious leaders throughout history. Gandhi was authentic, he was truly committed to his creed of Hinduism, and Hinduism in a very syncretic universal sense, not an exclusivist sense. He managed to combine his universality in politics with his universality in religion. The two words swaraj and satyagraha are examples of this. Swaraj is a vedic term, a term that Hindus used in the Vedanta to illustrate true emancipation, and emancipation was not merely political but there had to be a spiritual emancipation as well and spiritual freedom means freedom from ignorance, freedom to know the truths of the unity between all being. Now that is a religious perception, the same way with satyagraha, it can't be described as merely a political tool, it speaks to a creed of not killing another, holding on to a truth, and that truth is the unity of all being. So in both cases, and any number of other cases swadeshi for example, our own country, or sarvodaya, the uplift of all, these are terms that have deep roots in religion.

Now some have argued that Gandhi's use of Hindu terms alienated the Muslims and so made partition more likely. I don't see it that way. I see him more trying to find Hindu terms and practices that would appeal to the Muslims, for example the fast. The fast can be used in a political sense, such as a filibuster or a hunger strike, but Gandhi also used it in a religious sense, that is one of self sacrifice where one purged oneself of enmity and anger. So the Calcutta fast is a perfect example of the admixture of religion and politics being linked together.

The Muslims, when Gandhi fasted in Calcutta and Delhi, were in a minority. They were being butchered by the majority Hindu population in most cities. Gandhi chose the locations of his fast to protect the Muslims and he charged the Hindu majority with the responsibility to protect the Muslims, they must not allow this minority to perish, in relation to ideas of swaraj and satyagraha. That was a powerful message. He was assassinated largely for the unifying ideas and success of these fasts. His assassin stated in his trial that he shot Gandhi because of his soft stance on Islam. He was overly tolerant of Islam and that if Gandhi had his way the British Raj would not be replaced by a Hindu Raj but by a secular Raj which would allow the Muslims political power, he didn't want that at all.

Again I stress the intertwining of religion and politics. Think of the assassination itself, here is a political figure, who is also a religious figure, being assassinated for religious reasons, which also apply to political reasons, that is, that the Muslims would get more political power in this new state. And the assassin is a devout Brahmin orthodox Hindu, and Gandhi as well claiming to be an orthodox Hindu. So an orthodox Hindu kills an orthodox Hindu, except that the assassin's interpretation of Hinduism is one of exclusivism and the other inclusivist.

Why is it that students are taught so much about violent leaders and not as much about great nonviolent leaders?

I wish that we could have that compelling force behind nonviolence that we have for violence. As we all know violence is deemed newsworthy, violence is the show. People want video games and films and all the rest to see performances of violence. Gandhi knew that so well, and that is why when he went on the 1930 Salt March he intensely interviewed the people that would go on the march with him. He chose strictly from his ashram, which were people who had undergone the training of nonviolence for over two years living with him there…Now why did he say that was so important? Because these people would not strike back when they were beaten. And subsequently when they walked up to challenge the British salt monopoly, when line after line of British brutally beat them down they did not strike back. Gandhi knew that if there was a single incident of violence on the part of the protesters then the news would get a hold of it and give the British more validity and reasons to strike back. He was very sensitive to this dynamic, and it had to be absolute.

King showed another example of this intense dedication to nonviolence when after the bombing of his house he stood on his porch and preached to the crowd to "love thy enemy". The moral force behind that makes it become newsworthy, even when the appeal of violence is not present. That manipulation of the media is the trick or accomplishment that somehow made nonviolence more newsworthy or more readily captured the public's imagination.

To make nonviolence newsworthy is a challenge. Gandhi did it with the Dandi Salt March. It was a drama, the best that one could imagine and he pulled off every act perfectly. There was no lapse in terms of the way in which he performed. King pulled it off beautifully with the Montgomery bus boycott. King noted that "people would look back and say I pulled a rabbit out of the hat with the Montgomery bus boycott and they will be waiting for me to do it again and again and again and it isn't that easy to do". But he managed to capture the imagination of the news at that time. One has to make the media want this kind of story.

Can you speak to the process in which Mohandas Karamchand Gandhi became Mahatma Gandhi?

Gandhi was immensely fortunate to have lived a long life, and as such we can see his life divided up into different phases. He was born October 2, 1869 and died January 30, 1948 so he lived 79 years. That allowed him to live a long, what I call, inclusivist phase. Before that phase we wouldn't have recognized him. As a young man he quickly became enamored of the British and British culture, to such an extent that, as he says in his autobiography, he wants "above all to become a British general and adopt all manners of the British". He takes violin lessons, elocution lessons,…he spends money he shouldn't have spent as he didn't have much during his time in London…on the best clothes from Bond street. He becomes a lawyer and dresses and acts as a dandy. That stage of his life changed dramatically after he went to South Africa.

In South Africa, he spent 21 years there, the real transformation between Gandhi the dandy into Gandhi the civil disobedient occurred. It occurred after he had tried everything, newspapers, petitions, court trials, to redress the grievances of the Indian community there (in South Africa) who were appalled and humiliated by apartheid. So he comes to the point where he cannot accept apartheid any longer and at the suggestion of another Indian in September 1906 Gandhi tried a change in tactics. He and a few others took a pledge to go to prison unless the law of apartheid is changed. Apartheid was not relinquished and he was imprisoned. After his release he staged his first march in 1913.

He had changed dramatically, and that change came about as a result of the brutality of apartheid regime. He goes back to India, and what changed him in India, improbably and unpredictably, was the Amritsar Massacre. In 1919 after the massacre he decides the British have lost it and that non-cooperation has to be escalated into a massive disobedience campaign. He stages his first disobedience campaign against the empire and says "now I am a rebel I am committed to the end of the British Raj". By 1946-47 the country has achieved its independence through nonviolence.

Are great leaders like Gandhi necessary for nonviolent movments to be successful?

In Bosnia I don't think they needed one. In South Africa they did a great deal before Mandela was released from prison. He was a charismatic leader but was in prison for 27 years. The leaders I have known and have studied have happened to be great charismatic leaders. I like the way in which Max Weber talks about charisma as value-neutral. Hitler was certainly among the most charismatic leaders that we have seen. Did Germany need that kind of charismatic leader in order to create the World War II/Holocaust? Some write persuasively that that could not have been done without Hitler, that he was absolutely crucial. That is not to say that Hitler was not the product of a number of circumstances, but the particular chemistry of the individual just lit up.

Martha Nussbaum makes the point that within every civilization there are clashes, contrary to Huntington's clash between civilizations, clashes between what she calls inclusivists and exclusivists, code words for the universalist/humanist versus the separatist/partisan. She also says that there is a clash within you and within me. That is we have our own forces to resolve. That was Gandhi's point; unless you resolve those forces you will have no capacity to be the kind of leader that is necessary for nonviolent revolution. Eric Ericson has written how Gandhi has struggled to overcome these. Gandhi would always say that his most difficult satyagrahas were within his own family.

Gandhi has been criticized for his treatment of women, can you talk a bit about your understandings of his views on women?

There are two schools of thought. One concerns Gandhi's treatment of his wife. In his autobiography he says he was a "cruel but kind husband" and makes comments about how his wife taught him satyagraha in the process of a pretty cruel and jealous interaction when they were kids—they both married at age 13. He comes across in the autobiography as negative in his attitudes toward women. This school also says that Gandhi tended to regard women as best in the home, not in the political scene.

Despite that, [the second school] says the empowerment and politicization of women was overwhelming in India during his campaigns. He managed to bring the women, during these campaigns, out of the home and to the political field. He did not object to this and they went to prison along with the men. Every time he spoke of women he said that they made better satayagris, practitioners of civil disobedience, than men because they have infinite patience and, this is where he gets in trouble, infinite capacity to suffer. These feelings were greatly inspired by his relationship with his own mother. It is understandable in the way we view it today, to see why seeing worth in women's capacity to suffer is going to arouse questions about sexism.

One of the best texts on this is by Sue Rudolf, who wrote on Gandhi's ability to mobilize women, especially how his feminine style of politics managed to bring Gandhi into the theater of real power. That feminine style of politics was characterized by self-sacrifice, patience, compassion, qualities that we do often associate, stereotypically or not, with women. He saw the masculine macho model as set forth by the British, but also by Indian political groups, as a real obstacle. He wanted to get across the image of compassion that he had seen in his mother. He will continually refer to the influence of his mother. Should he be called a feminist? On one hand this is problematic; on the other hand one of the great feminists in India, Madhu Purnima Kishwar, says he was "a champion of women because he brought them out of the home and into the political theater".

How is Gandhi relevant to the world today, why should we read him?

As Nussbaum says there is a clash within everyone. We read Gandhi to see how nonviolence has power. We read him to reinforce that element within ourselves that is nonviolent, which speaks to compassion, speaks to non-domination, patience, understanding, empathy. When I teach Gandhi I try to convey this above all, that we take him as a political theorist for his practice of nonviolence. If it were not for his practice, for his demonstration of the power of nonviolence then he would be nowhere. He says that himself, "If you want to learn from me look to my life experience my example. My teachings will vary, I may contradict myself but my commitment to nonviolence is steady and sure, consistent".

This culture is fraught with violence, we are bombarded with violent images, language, symbols. We don't have people today leading nonviolent movements. It is only in education that we can nurture the forces of nonviolence within us, to try to elicit the qualities of empathy and understanding that we want to see in our students, children, and people all around us.

Monday, May 18, 2015

DIY PDF to Excel / Spreadsheet conversion

1. If you want to do multiple pages at a go, then you'll first need to extract just those pages from the bigger document. For that, you can use:

2. Then, this site will convert that pdf to excel for you.. it was pretty accurate for me:
If not that, then my second choice is: 
You might have to clean up the converted excel, re-align stuff, etc.

3. In case it's just a few pages, or if you need a specific part of the page only, then better to use Tabula: 
It's a portable software that runs on your computer, works through your browser. You visually draw a box on the page and it converts that to table. Either copy it to clipboard and then paste in excel using import text wizard, or save it as CSV and then load it in excel.

4. Now, the converted file may have weird characters, if your document was using legacy fonts like Shree-Dev to display vernacular language data.
Here's a site having converters to convert such text to Unicode format, which can then be read, copy-pasted, searched:
You'll get HTML files that have scripts in them. Save them locally and run them in your browser. (note: certain browsers might block the script from working unless you activate it.)

For Pune's Budget book, which was using a font similar to but slightly different from Shree-Dev, I made a customized converter, you can copy it from here:
Convert, copy, and in excel, select just the topmost cell of your selection and press ctrl+v. all the cells you originally copied out from should be replaced.

In case this was in regular ShreeDev font, try this:

The text might get garbled up in the pdf-to-excel conversion. Try this mid-development version of Tabula:
For more details, see this thread: . It handles legacy Devnagri scripts and does the conversion perfectly without distorting.
(Not confirmed but might work better with other Indian scripts too. But doesn't work with Unicode well the same way all others don't.. that seems to be an inherent problem in PDF technology.)

If is still happens (happened a lot on my end and later I got Tabula devs to fix it), then you have to copy-paste the text from the pdf. Press and hold the Ctrl, Alt buttons and draw a box (box-select) in the pdf-reader, that should let you select all the text in a column. First paste that in a simple text editor like Notepad.

Where there was more than one line of text in a cell, you'll have to "unwrap" them back to one line. (switch off wrapping by Format > Word Wrap). Where there were blank cells, you'll have to press enter, enter to space out the lines. Once you're sure it's exactly as it was in the PDF, then you can safely paste it back to the excel. Use the adjoining cells to make sure the stuff is well aligned.

5. For the figures, if they have commas like this: 52,50,000 then they'll be stored in the excel as text, not as numbers. To make them as numbers, select all the cells having the numbers, copy, and paste here:
..and press "zap commas". Copy the output and paste it back to the excel, with the first cell of your copied block selected.
(you can look for other such converters too or just make your own.. my file's source code is here:

Tuesday, May 12, 2015

Food as humanity's primary renewable energy source

I sense opportunity in changing the narrative to count as energy and technology, things that haven't been considered as so by the mainstream.

Food is solar energy. Energy from the sun is converted to a medium that powers living beings to move and grow and live. It still has far higher efficiency than PV, wind etc and is a storage medium.

Permaculture / organic farming is a technology that removes the fossil fuel dependence of food energy. Just because it may not immediately involve silicon chips and wires or handling of poisonous substances doesn't necessarily make this a non-technology.. We can instead look at its sophistications. Wind direction, terrain, light and shade, companion plants, 100% waste recycling, macro and micronutrients management through influencing soil microbes and beneficial insects (coming a long way from the simplistic N-P-K and kill-everything-else doctrines), designs that incorporate water management, location-specific micro-climate adaptation ... It takes farming to totally new levels of sophistication and productivity potential that make conventional fertilizer-pesticide farming and its predecessors look crude, backward and uncivilized.

To top it off, this technology has been proven to not just avoid or reduce CO2 emissions (which is the primary focus of, say, solar PV), but it actually reverses them, sequestering carbon back into the ground. So imagine a solar panel that while providing you with energy that's inherently storage-friendly, also pulls excess CO2 out of the atmosphere, and does not use rare earth metals or have a pile of embodied costs. That's permaculture technology. (Yes there are many other names of this but I suggest we just go with the one that more than one nationality's peoples are already familiar with)

So yes, do keep eyes open for renewables technologies, even forms that haven't been taken into account as yet by a mainstream that chooses to ignore innovations in the world's largest and most primary energy sector (food) when talking about energy.

PS: all technologies have a transition period, with early adopters investing more, feedback cycles for improvement, not getting immediate ROI, time taken for standardization, lack of available resources, learning curve, etc and the one mentioned is probably no different.

Monday, May 4, 2015

Breaking through problem-reaction-solution traps

Here's an analytical/critical thinking experiment : It would help if
the following were detailed out:

1. What misunderstandings are there in the analysis of the problem?

2. What conclusion has been arrived upon and how it's problematic,
what would be the consequences if that conclusion were put into
effect? Specify how the misunderstandings if any are leading to a

3. In which stakeholders' interest is the advocacy being done in. Are
they meant to be stakeholders here to begin with, or are they just an
excuse? Are their interests being truly represented, and is the
advocated solution a true solution for them?

4. Which stakeholders' interests are being ignored?

5. Is there any common ground between the two stakeholders groups
mentioned, are there any similar goals, intentions or problems
affecting both?

6. Is there any genuine reason for the two stakeholder groups to be
pitted against each other?

7. Are there any background intentions behind what has been published,
any stakeholders who aren't being mentioned that would surreptitiously
benefit should the proposed changes be brought about? (example:
textbook publishers, private education industry, etc) Who gains if the
misunderstandings persist?

8. What is an alternative change to the status quo that rectifies
misunderstandings, if any, and serves the interests of both the
stakeholder groups mentioned

9. If the status quo is not to be changed and things are fine as they
are, then what are the clarifications that will clear away the
misunderstandings that led to a perception of a problem being there
and needing redress? Is there a motive for any stakeholder to maintain
the misunderstanding?

Friday, May 1, 2015

Hints to an aerodynamic model possibly more efficient than today's regular planes

Built scaled up replicas of gold trinkets recovered from tombs, put propellers on them, and successfully flew them.
" At the August, 1997 meeting of the " Ancient Astronaut Society World Conference," gathered in Orlando, Florida, the two researchers presented their independent findings on the "insects." Those assembled literally cheered when Belting's model (powered by a small jet engine - left) flew a set of complex aerodynamic maneuvers … and then came in for a perfect landing - all with a model airplane built as an exact match to a tiny gold trinket found in an Incan grave … dated minimally to over a thousand years ago!"

It's quite healthy to get one of these googly's now and again :) Just imagine the far more logical and practical approach that would have been followed to properly test this stuff out if the mainstream stopped resorting to all the "conspiracy theory" branding and dismissing and instead just took things logically and allowed people to explore. Instead, no, we have to brand this and dismiss it and ridicule the people interested in it and so cut out any possibility of actually testing and proving or disproving.

The legalese it takes to destroy democracy

"Protection of persons acting in good faith under this Act from prosecution, suit or other legal proceedings, except with the sanction of the Central Government, in exercise of the powers conferred by this Act."

That's all the legalese it takes to destroy democracy in your country, for the sake of a "greater good" that the guys with the guns can decide upon. Stick this line at the end of any law, and you've effectively disabled anyone badly affected by that law from seeking any justice or relief or remediation. In the absence of legal avenue for justice, if you dare to seek it using any other means, even civil disobedience, then you're branded as terrorist or uncivilized or anarchist or disrespectful of due processes and not worthy of any justice. Our ever-ready army of educated intellectuals who don't bother fully reading what they are defending or checking what it actually translates to on the ground, will gladly dismiss you off while remaining in blissful ignorance of the fact that the law itself bans you from pursuing the correct avenues for justice. 

Got raped by a policeman acting under this Act? Got your land taken away just before harvest time? Sorry, bitches, you've been deprived. All in "good faith", of course.

Interesting links for March and April 2015

So here comes another omnibus links dump! I missed the passing of March into April, and now we're in May. I got busy with some work and discontinued putting in interesting links, so missed a lot of things. Still, this collection is awesome.

Sharing tools and best practices for budget data analysis

Yes, I'm bad at sharing what I myself am up to. But for some reason when communicating with a person it happens nicely. So, forwarding an email:


2 open source tools that were used:

PDFtk4all : for splitting heterogenous pdf's into homogenous sections, so that each file only has one kind of table.

Tabula : for refined conversion from PDF to excel.

Go down this thread : to get a dropbox link to a version that handles Indian scripts and does the conversion perfectly without distorting. (they still have to push the update to the main version.) (The conversion happens nicely if the PDF uses legacy fonts, that is. If the data is in unicode, then God help us. Whatever you do, please don't persuade governments to publish large tables in pdf in unicode! If they are switching to unicode, then they should stop making PDFs of the docs and just give a link that we can compare the file with for authenticity.)

Our biggest hurdle is usually the with getting the data in Unicode format.
Here is a link to the converter I used:

That one was for a particular font that the PMC uses.

For Shree-Dev, which is a more popular font and which the bus authority here uses, here's the converter:

Detail: There's a substitution array of about 200 characters that we usually have to map to their unicode equivalents. The converter scripts basically does a quick find+replace on all the substitution pairs. But in Hindi/Marathi we also have vowel "maatras" and a variety of combos, and that can get a little tricky, hence some scripting is needed. I didn't make the script.. I got it from an online group and adapted it to work with what I had.

An example of letter-substitutions seen across some popular legacy fonts:


Also, I've recently been working on using mapping to facilitate budget analysis. Here's one outcome, for browsing through this year's Pune's Participatory Budget:

One significant breakthrough here was learning how to combine geolocation data with a separate excel file having the info we want the map to show on the marked areas. More on how that happened is partially explained here:

And.. did you know google spreadsheets turns our regular graphs into interactive HTML5 ones? We can embed them on website, like so: (note: website is work in progress and this is a temporary address.)

All the visualizations you'll see in the pages here are coming from google spreadsheets, just like excel, making graphs out of table data. No coding needed; I just had to copy-paste the html code.

There's some quite nice open data sharing platforms out there, example: etc. I'm going to explore how to plug into them.

Gift Economy

Would you like to show your appreciation for this work through a small contribution?

(PS: there's no ads or revenue sources of any kind on this blog)

Related Posts with Thumbnails