Non-affine groups acting doubly transitively on a Hadamard matrix have been classified by Ito. Implicit in this work is a list of Hadamard matrices with non-affine doubly transitive automorphism group. We give this list explicitly, in the process settling an old research problem of Ito and Leon. We then use our classification to show that the only cocyclic Hadamard matrices developed from a difference set with non-affine automorphism group are those that arise from the Paley Hadamard matrices. If H is a cocyclic Hadamard matrix developed from a difference set then the automorphism group of H is doubly transitive. We classify all difference sets which give rise to Hadamard matrices with non-affine doubly transitive automorphism group. A key component of this is a complete list of difference sets corresponding to the Paley Hadamard matrices. As part of our classification we uncover a new triply infinite family of skew-Hadamard difference sets. To our knowledge, these are the first skew-Hadamard difference sets to be discovered in non-abelian p-groups with no exponent restriction. As one more application of our main classification, we show that Hallʼs sextic residue difference sets give rise to precisely one cocyclic Hadamard matrix.