++, last & init faster than :, head & tail?

Question

Given these two ways of writing a function that finds all primes up to a specific number:

primes1 = iterate
    (\ps -> ps ++ [([x |
        x <- [last ps + 1..],
        all (\p -> x `mod` p /= 0) ps] !! 0)])
    [2]

primesTo1 :: Integer -> [Integer]
primesTo1 n = init $ head $ dropWhile (\p -> last p <= n) primes1

primes2 = iterate
    (\ps -> ([x |
            x <- [head ps + 1..],
            all (\p -> x `mod` p /= 0) ps] !! 0)
        : ps)
    [2]

primesTo2 :: Integer -> [Integer]
primesTo2 n = tail $ head $ dropWhile (\p -> head p <= n) primes2

Why is primesTo1 a lot faster than primesTo2, even though the different functions used; primesTo1 uses ++, last & init instead of :, head & tail used in primesTo2.

Output of ghci with :set +s:

*Main> primesTo1 10000
...
(0.51 secs, 124779776 bytes)
*Main> primesTo2 10000
...
(3.30 secs, 570648032 bytes)

Compiled with ghc -O2:

$ time ./primes1
...
./primes1  0.06s user 0.00s system 68% cpu 0.089 total
$ time ./primes2
...
./primes2  0.28s user 0.00s system 98% cpu 0.283 total

---

Note: I'm not looking for an optimal prime number generator for Haskell, I'm just confused by the speed differences of the two functions.

Answer 1

As pointed out by the "n.m.", the reason for this is that primes2 first tries to divide by the largest found primes, while primes1 starts with the lowest.

So first reversing the list of current primes, before using them in all is actually faster than both primes1 and primes2:

primes3 = iterate
    (\ps -> ([x |
            x <- [head ps + 1..],
            all (\p -> x `mod` p /= 0) $ reverse ps] !! 0)
        : ps)
    [2]

primesTo3 :: Integer -> [Integer]
primesTo3 n = tail $ head $ dropWhile (\p -> head p <= n) primes3

ghci speed with 10000 as argument:

*Main> primesTo3 10000
...
(0.41 secs, 241128512 bytes)

and compiled with ghc -O2:

$ time ./primes3
...
./primes  0.05s user 0.00s system 24% cpu 0.209 total

Answer 2

I know you said you're "not looking for an optimal prime number generator for Haskell" but you were still interested in "speed differences of the two functions". So here's few more syntactic manipulations on your corrected function, primes3, which adds primes by (:), in reversed order, and reverses them for testing each time,

primes3 :: [[Integer]]
primes3 = iterate
    (\ps -> ([x |
            x <- [head ps + 1..],
            all (\p -> x `mod` p /= 0) $
                takeWhile (\p-> p*p <= x) $ reverse ps] !! 0)
        : ps)                            -- ^^^^^^^^^^
    [2]

This code can be modified (although this won't change the efficiency):

primes3b :: [[Integer]]
primes3b = iterate
    (\ps -> ([x |
            x <- [head ps + 1..],
            all (\p -> x `mod` p /= 0) $
                takeWhile (\p-> p*p <= x) $ map head $ primes3b ] !! 0)
        : ps)                            -- ^^^^^^^^^^^^^^^^^^^
    [2]

isn't it? and that is equivalent to (notice the type change)

primes4 :: [Integer]
primes4 = map head $ iterate
    (\ps -> ([x |
            x <- [head ps + 1..],
            all (\p -> x `mod` p /= 0) $
                takeWhile (\p-> p*p <= x) primes4 ] !! 0)
        : ps)                          -- ^^^^^^^
    [2]

which is the same as

primes5 :: [Integer]
primes5 = iterate
    (\p -> head [x | x <- [p + 1..],
                 all (\p -> x `mod` p /= 0) $
                   takeWhile (\p-> p*p <= x) primes5 ] 
           ) -- nothing!
    2

or, a little bit faster,

primes6 :: [Integer]
primes6 = 2 : iterate
    (\p -> head [x | x <- [p + 2, p + 4..],
                 all (\p -> x `mod` p /= 0) $ tail $
                   takeWhile (\p-> p*p <= x) primes6 ] )
    3           -- ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Lastly, (and with considerable gain in speed and even empirical orders of growth it turns out) - why add just one number at each iteration, having spent work to fetch the primes to test by? We can work by portions between consecutive squares of primes. The fetched primes list is of the same length for these segments, and that length increases by 1 from segment to segment:

primes7 :: [Integer]
primes7 = concatMap snd $ iterate 
              (\((n,p:ps@(q:_)),_) -> ((n+1,ps),
                       [x | let lst = take n $ tail primes7,
                            x <- [p*p + 2, p*p + 4..q*q-2],
                            all ((/= 0).rem x) lst]))
              ((1,tail primes7),[2,3,5,7]) 

which actually express the optimal trial division sieve.