Redes, Python e Big Data

Graphs in the database. SQL meets social networks – A teoria de grafos está na moda e naturalmente aplicações que recorram ao poder do grafo social precisam de bases de dados que suportem a estrutura das relações subjacentes. Interessante artigo sobre como fazê-lo…

Python in Big Data with an overview of NumPy & SciPy – Python é lento!… Quantas vezes já ouvi isto… Um vídeo a não perder.

Algumas Alternativas ao Matlab

O Matlab é o cavalo de batalha de muitos cientistas e engenheiros, mas há alternativas que podem ser exploradas. Aqui ficam algumas alternativas ao Matlab sem nenhuma ordenação em particular.

Scilab

Octave

JMathLib – Um clone Java do Octave, SciLab e Matlab

TeLa the Tensor Language

Algae

Lush (Lisp Universal SHell)

Yorick

Rlab

Maxima

Euler

S-Lang library

Python com NumPy e SciPy

The R Project for Statistical Computing – um dos meus favoritos!

Conclusão

A escolha de cada um vai depender muitas vezes de outros factores que não simplesmente as características técnicas de cada programa, sendo que a integração destas ferramentas no ciclo de trabalho normal do cientista e da sua equipa, do tempo para aprendizagem do novo programa/linguagem ou a possibilidade de suporte local ou remoto, poderão eliminar algumas opções. A verdade é que as ferramentas existem, o interessante é que se faz com elas.

Tim, can I have an USB port on my iPad, please?

One thing that I really think lacks on the iPad is an USB port. I don’t really want to connect any fancy thing to it, just a stupid usb flashdrive so I could share some documents to other people when I’m on the road and don’t have internet access.

Is that so difficult? I could even consider buying an external adapter (paying the customary 39$ that apple charges for those iPad adapters) just to be able to plug in a pen…

A simple 2GB pen could act as external storage and apps could use them as ‘send to’ location. That would without a doubt allow the iPad to truly replace my laptop in many field trips with colleagues.

And although what I want is a very simple thing, imagine the possibilities that would open up for all other USB devices out there! Imagine that, think different!

If apple is trying to merge Desktop computing and Mobile computing, apple needs to mix what is prevalent and important on both sides, and storage is probably the most important thing, and it can’t be fully replaced by cloud storage, at least not when there’s no cloud.

2 upgrades – site estático e markdown

Markdown

Basicamente prefiro utilizar Markdown para escrever os meus posts (e não só) a qualquer editor visual é completamente irritante e normalmente o meu processo de escrever os posts era feito offline em Markdown e depois convertido para HTML que era depois colado no post. Fartei-me deste processo e decidi utilizar um plugin para o WordPress de forma a fazer bypass ao processo offline. Descobri o excelente WP Markdown e agora estou no paraíso!

Blog estático

Outro aspecto que sempre quis implementar no blog era o de servir páginas estáticas em vez de dinâmicas. O primeiro passo foi eliminar os comentários internos e mudá-los para o Disqus e seguidamente ter uns scripts que corriam via cronjob e sempre que postava algo novo. O processo era um pouco lento e impedia que utilizasse clientes xmlrpc directamente (tipo iPad ou MarsEdit). Agora finalmente escrevi um plugin do wordpress (os que encontrei não me agradaram) para resolver o problema o que facilita a utilização de geração de páginas estáticas para o site. Para já o plugin ainda está um pouco verde mas se houver interessados em saber como o implementei digam qualquer coisa nos comentários.

Collusion – Quem está a fazer tracking dos teus movimentos?

Ultimamente tem surgido alguns reports que as grandes companhias andam a fazer tracking das navegações dos utilizadores de forma algo abusiva. Desde sempre que perceber os hábitos de navegação dos utilizadores é uma fonte muito importante de rendimentos para as empresas, mas entretanto encontrei uma ferramenta chamada Collusion que funciona para Firefox que é muito interessante por representar esses tracking cookies na forma de um grafo. Uma pessoa fica assustada com a quantidade de cookies que são colocados despejados no nosso browser… experimentem!

Comparing JavaScript loops performance (for, do while, for in)

update:The Chrome results where so strange that I retook the tests again with more samples and averaged the results. Things look more reasonable now.

I always try to optimise code for the fastest speed possible. While playing with some javascript (the culprit is this marvelous game by @tonyvirtual called Chain Reaction) I started thinking about the different speed gains that one might get from using different kinds of loops. A basic for loop in javascript can be implemented in many ways but typical are these 3:

// Loop 1
for (var i=0; i < Things.length; i++) {
    // Do something with Things[i]
};
 
// Loop 2
var i = Things.length-1;
do {
// Do something with Things[i]
} while (i--);
 
// Loop 3
for (var i in Things){
// Do something with Things[i]
}

Your Javascript results

Previous Results with other browsers

There are many online reports that favour one of the loops instead of the others, but I wasn’t convinced so I decided to test them myself. I devised a simple test that you can run in your browser or you can modified it by getting it from github.com and tested the script in 3 different browsers all on the Mac OS X 10.5.8. The browsers are Firefox 10.0.2, Safari 5.0.6 and Chrome 19.0.1049.3 dev so I don’t know if these will apply to other browsers or OSes (use the github file).

The JavaScript performance results are listed below in the format min (average 10x):

Firefox

Loop 1 (for ;;) — 117 (118.5)
Loop 2 (do – while) — 119 (125.9)
Loop 3 (for in) — 600 (671.6)

Safari

Loop 1 (for ;;) — 180 (185.8)
Loop 2 (do – while) — 178 (183.3)
Loop 3 (for in) — 1591 (1615.9)

Chrome

Loop 1 (for ;;) — 180 (191.2)
Loop 2 (do – while) — 159 (167.6)
Loop 3 (for in) — 585 (610.2)

JavaScript traditional for loop is the fastest

I am really surprised with these results. I knew that Mozilla had made some trick to make for loops fast, but I didn’t imagine that Firefox would beat the other two. Also the for in is to be avoided as it is slower. Convenience in this case comes at a price. I also was amazed on how bad the do-while loop performed under Chrome. I retook the test for Google Chrome and it now makes sense, the do-while is faster than the traditional one but still slower than Firefox’s.

I don’t understand this JavaScript performance differences but they certainly need more testing. The only conclusion until now is that the traditional javascript for loop is fast and should be used without second thoughts.

function go(){
    var o = document.getElementById('out');	
 
    var ar = new Array();
    for (var i=0; i < 1000000; i++) {
        ar[i]=i;
    };
 
    o.innerHTML+="Loop 1 (for ;;) ";
    var start =  (new Date()).getTime();
    for (var i=0; i < ar.length; i++) {
        var a = Math.random()*i;
        var b= Math.random()*i;
        var c= Math.round(a-b);
        var d= Math.round(a)-Math.round(b);
    }
    var end = (new Date()).getTime();
    o.innerHTML+="-- "+(end-start);
 
 
    o.innerHTML+="<br/>Loop 2 (do - while) ";
    var start =  (new Date()).getTime();
    var x = ar.length-1;
    do {
        var a = Math.random()*x;
        var b= Math.random()*x;
        var c= Math.round(a-b);
        var d= Math.round(a)-Math.round(b);			
    } while (x--);
    var end = (new Date()).getTime();
    o.innerHTML+="-- "+(end-start);		
 
 
    o.innerHTML+="<br/>Loop 3 (for in) ";
    var start =  (new Date()).getTime();
    for (var i in ar) {
        var a = Math.random()*i;
        var b= Math.random()*i;
        var c= Math.round(a-b);
        var d= Math.round(a)-Math.round(b);
    }
    var end = (new Date()).getTime();
    o.innerHTML+="-- "+(end-start);		
}

Evernote needs Markdown support

Evernote needs Markdown support

I love Evernote. It’s one of my favourite apps for taking notes. I use it either on the desktop and on the iPad. But I also love Markdown as it is the best language for writing for the web. Hell, Markdown should have been the original markdown of the web in the first place. Thus I grief when Evernote doesn’t have support for Markdown.

Markdown in Evernote todo list:

  1. Automatic syntax highlight. Although not important, it is a feature that I love in my Emacs Markdown mode.
  2. An export feature that allows me to export a markdown note to MD and HTML files. Also, a preview feature would be nice sometimes you forget that link text comes first than the url and a simple preview would solve that problem. Finally, a ‘send to blog’ would be cool.
  3. Emacs spoiled me. I love the fill-mode and it is surprising that so few editors have a similar feature. Most text editors implement some sort of Soft/Hard word wrapping, but none as done it in the same way as Emacs. Emacs fill-mode is perfect for Markdown.

These are 3 things instead of 2 about Markdown in Evernote, but I thought that I’d throw the last one as wishful thinking…