LinuxQuestions.org
Visit Jeremy's Blog.
Home Forums Tutorials Articles Register
Go Back   LinuxQuestions.org > Forums > Non-*NIX Forums > Programming
User Name
Password
Programming This forum is for all programming questions.
The question does not have to be directly related to Linux and any language is fair game.

Notices


Reply
  Search this Thread
Old 05-07-2021, 01:29 PM   #106
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 22,101

Rep: Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365

Quote:
Originally Posted by mimorek View Post
PathFinderCircleOfSquares.py N finds all valid paths of sequence 1..N.

uniqueCycles.py tests all found paths for uniqueness and prints the unique circles. It removes duplicates and mirrors.
profiling again for N=40:
Code:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000  100.604  100.604 <string>:1(<module>)
        1   77.604   77.604  100.604  100.604 h.py:20(run)
        1    0.000    0.000    0.000    0.000 h.py:66(<listcomp>)
        1    0.000    0.000  100.604  100.604 {built-in method builtins.exec}
383766648   13.412    0.000   13.412    0.000 {built-in method builtins.len}
     1180    0.016    0.000    0.016    0.000 {built-in method builtins.print}
 54219386    2.074    0.000    2.074    0.000 {method 'append' of 'list' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
 54219256    4.707    0.000    4.707    0.000 {method 'index' of 'tuple' objects}
        1    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {method 'keys' of 'dict' objects}
 54219256    2.791    0.000    2.791    0.000 {method 'pop' of 'list' objects}
you can see the method len was invoked a lot, almost 7 times more than anything else.
A small modification:
Code:
    # while not (len(path) == 1 and a == term):  # instead of this
      while not (a == term and len(path) == 1):  # in line 48
will significantly improve the speed (15 %):
Code:
   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
        1    0.000    0.000   85.986   85.986 <string>:1(<module>)
        1   67.774   67.774   85.986   85.986 h.py:20(run)
        1    0.000    0.000    0.000    0.000 h.py:66(<listcomp>)
        1    0.000    0.000   85.986   85.986 {built-in method builtins.exec}
275694526    9.083    0.000    9.083    0.000 {built-in method builtins.len}
     1180    0.015    0.000    0.015    0.000 {built-in method builtins.print}
 54219386    1.979    0.000    1.979    0.000 {method 'append' of 'list' objects}
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
 54219256    4.464    0.000    4.464    0.000 {method 'index' of 'tuple' objects}
        1    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {method 'keys' of 'dict' objects}
 54219256    2.672    0.000    2.672    0.000 {method 'pop' of 'list' objects}
just because we could save more than 100M calls to len.
I guess we can improve more again by storing that len in an additional variable and follow path instead of recalculating length. Probably I will try that later....

From the other hand I like that python implementation very much.
 
1 members found this post helpful.
Old 05-07-2021, 01:43 PM   #107
mimorek
Member
 
Registered: Feb 2013
Distribution: Debian (jessie)
Posts: 42

Rep: Reputation: Disabled
Quote:
Originally Posted by igadoter View Post
No matter starting point? I mean shift left/right? In circle 1,2,3 and 2,3,1 are the same.
Yes. I think so.

Code:
$ cat | ./uniqueCycles.py
1 2 3 4
1 2 4 3
1 3 2 4
1 3 4 2
1 4 2 3
1 4 3 2

 1  2  3  4            #
 1  2  4  3            # These three are unique.
 1  3  2  4            # 
Paths in input:  6
Unique cycles: 3

Last edited by mimorek; 05-07-2021 at 05:28 PM.
 
Old 05-07-2021, 05:03 PM   #108
mimorek
Member
 
Registered: Feb 2013
Distribution: Debian (jessie)
Posts: 42

Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
you can see the method len was invoked a lot, almost 7 times more than anything else.
A small modification:
Code:
    # while not (len(path) == 1 and a == term):  # instead of this
      while not (a == term and len(path) == 1):  # in line 48
will significantly improve the speed (15 %):
...
...
just because we could save more than 100M calls to len.
I guess we can improve more again by storing that len in an additional variable and follow path instead of recalculating length. Probably I will try that later....
I tried your improvement:
Code:
$ time ./PathFinderCircleOfSquares.py 40 >/dev/null
real    3m41.601s
user    3m41.468s
sys     0m0.016s

$ time ./improved_PathFinderCircleOfSquares.py 40 >/dev/null        
real    3m33.675s
user    3m33.526s
sys     0m0.008s
Is a 3.6 % time reduction on my computer, but nonetheless.

Quote:
Originally Posted by pan64 View Post
From the other hand I like that python implementation very much.
Thanks.

What did you use for the profiling pan64?
 
Old 05-08-2021, 05:47 AM   #109
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 22,101

Rep: Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365
that was the default python3 profiling module, cProfile. Nothing else. It looks like profilers will influence timings .... Or probably depends on the version of python?
And what about this:
Code:
# Find circle

path = [1]
a = 0
term = len(possq[1])

while not (a == term and len(path) == 1):
    while True:
        try:
            c = possq[path[-1]][a]
            if c not in path:
                path.append(c)
                a = 0
                if len(path) == high:
                    print(*path, sep=' ')
                break
            a += 1
        except:
            b = path.pop()
            a = possq[path[-1]].index(b) + 1
            break
 
Old 05-08-2021, 07:34 AM   #110
mimorek
Member
 
Registered: Feb 2013
Distribution: Debian (jessie)
Posts: 42

Rep: Reputation: Disabled
Quote:
Originally Posted by pan64 View Post
that was the default python3 profiling module, cProfile.
Thanks. Found it.

That's an interesting use of try except.
I thought it was kind of "expensive" to use try except. I wouldn't have thought of using it. But it works.
Quote:
Originally Posted by pan64 View Post
And what about this:
Code:
# Find circle

path = [1]
a = 0
term = len(possq[1])

while not (a == term and len(path) == 1):
    while True:
        try:
            c = possq[path[-1]][a]
            if c not in path:
                path.append(c)
                a = 0
                if len(path) == high:
                    print(*path, sep=' ')
                break
            a += 1
        except:
            b = path.pop()
            a = possq[path[-1]].index(b) + 1
            break
Are you catching the IndexError in this line?
Code:
            c = possq[path[-1]][a]
Testing:
Code:
$ time ./PathFinderCircleOfSquares.py 40 >/dev/null        
real    3m33.675s
user    3m33.526s
sys     0m0.008s

$ time ./pan64_PathFinderCircleOfSquares.py 40 >/dev/null   # With try except.
real    3m15.360s                                           # That's a time saver!
user    3m15.265s
sys     0m0.004s
 
Old 05-08-2021, 08:45 AM   #111
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 22,101

Rep: Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365
Quote:
Originally Posted by mimorek View Post
Are you catching the IndexError in this line?
Code:
            c = possq[path[-1]][a]
Yes, that will raise an exception if a was too high. Identical [more or less] to
Code:
while a < len(possq[path[-1]]):
But do not need to calculate the condition, it will automatically break the loop. Probably this is even better:
Code:
    while not (a == term and len(path) == 1):
        try:
            while True:
                c = possq[path[-1]][a]
                if c not in path:
                    path.append(c)
                    a = 0
                    if len(path) == high:
                        print(*path, sep=' ')
                    break
                a += 1
        except:
            b = path.pop()
            a = possq[path[-1]].index(b) + 1

Last edited by pan64; 05-08-2021 at 08:49 AM.
 
Old 05-08-2021, 01:48 PM   #112
pan64
LQ Addict
 
Registered: Mar 2012
Location: Hungary
Distribution: debian/ubuntu/suse ...
Posts: 22,101

Rep: Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365Reputation: 7365
Creating two matrices:
Code:
m = [ [0]*high ]
mr = [ [0]*high ]
for key in possq.keys():
     l = possq[key]
     while len(l) < high:
         l.append(0)
     m.append(l)

     k = [0]*high 
     for i in range(high):
         if l[i]:
             k[l[i]-1] = i+1
     mr.append(k)
mr is a reverse lookup matrix to speed up index calls
the loop will be quite simple:
Code:
def circle():
    path = [1]
    a = 0
    term = len(possq[1])

    while not (a == term and len(path) == 1):
        while True:
            c = m[path[-1]][a]           # = possq[path[-1]][a]
            if c == 0:
                b = path.pop()
                a = mr[path[-1]][b-1]    # = possq[path[-1]].index(b) + 1
                break;
            if c not in path:
                path.append(c)
                a = 0
                if len(path) == high:
                    print(*path, sep=' ')
                break
            a += 1
And I think it is really definitely faster now.
And if you use numpy martices it will be much slower.....
 
Old 05-08-2021, 08:55 PM   #113
mimorek
Member
 
Registered: Feb 2013
Distribution: Debian (jessie)
Posts: 42

Rep: Reputation: Disabled
Wow that is amazing!
post6249286
Quote:
Originally Posted by pan64 View Post
And I think it is really definitely faster now.
Yes it is!

Code:
$ old_circleOfSquares.py 40
real    3m9.611s
user    3m9.483s
sys     0m0.041s

$ new_circleOfSquares.py 40
real    2m33.655s            # wow :)
user    2m33.544s
sys     0m0.053s
I'm still trying to work out what is going on in your code, but the difference is significant. Great job!
 
Old 05-09-2021, 02:59 AM   #114
floppy_stuttgart
Senior Member
 
Registered: Nov 2010
Location: EU mainland
Distribution: Debian like
Posts: 1,156
Blog Entries: 5

Rep: Reputation: 107Reputation: 107
Quote:
Originally Posted by dogpatch View Post
2. Each element in the ring must be within the defined limits. In this case, between 1 and 32, with each number occurring exactly once (in any order).
Hello, please share with your mathematician friend my view that 1 and 2 are not "numbers" but "functions" (additionally to "+" "-" "*"). Perhaps it will making thinking differently in his problem.
1: is an "Increment" (some others were representing/using this as a dirac impulse) in sense of combination with "+" and "invariant" in sense of use with "*"
2: is a "Doubling" in the sense of "elliptical" thinking in combination with "*"
And I would be interested to see the result of some "circles" by taking them out.

Last edited by floppy_stuttgart; 05-09-2021 at 03:04 AM.
 
Old 05-09-2021, 03:13 AM   #115
igadoter
Senior Member
 
Registered: Sep 2006
Location: wroclaw, poland
Distribution: many, primary Slackware
Posts: 2,717
Blog Entries: 1

Rep: Reputation: 625Reputation: 625Reputation: 625Reputation: 625Reputation: 625Reputation: 625
Quote:
Originally Posted by floppy_stuttgart View Post
Hello, please share with your mathematician friend my view that 1 and 2 are not "numbers" but "functions" (additionally to "+" "-" "*"). Perhaps it will making thinking differently in his problem.
1: is an "Increment" (some others were representing/using this as a dirac impulse) in sense of combination with "+" and "invariant" in sense of use with "*"
2: is a "Doubling" in the sense of "elliptical" thinking in combination with "*"
And I would be interested to see the result of some "circles" by taking them out.
Probably it is about me. Here in this thread there is c program. With help of it I generated amazing circle containing ca 24 thousands of numbers. And essentially I was little bored to look for larger sequences. The problem is of mathematics of course - but here main point is set on programming tools. About your concept numbers to be functions please elaborate little more.
 
Old 05-09-2021, 03:09 PM   #116
mimorek
Member
 
Registered: Feb 2013
Distribution: Debian (jessie)
Posts: 42

Rep: Reputation: Disabled
@pan64

Quote:
Originally Posted by pan64 View Post
Creating two matrices:
Code:
m = [ [0]*high ]
mr = [ [0]*high ]
for key in possq.keys():
     l = possq[key]
     while len(l) < high:
         l.append(0)
     m.append(l)

     k = [0]*high 
     for i in range(high):
         if l[i]:
             k[l[i]-1] = i+1
     mr.append(k)
I like the way how you've used 0 to control the loop. Saves a lot of len() calls.

I changed your lookup lists into dictionaries. That sped things up some more.

Code:
$ time old_circleOfSquares.py 40 >/dev/null
real    2m33.655s            
user    2m33.544s
sys     0m0.053s

$ time new_circleOfSquares.py 40 >/dev/null
real    2m6.746s
user    2m5.758s
sys     0m0.168s
The latest version is attached to this message.
Attached Files
File Type: txt circleofsquares_v02.txt (1.9 KB, 15 views)
 
Old 05-09-2021, 06:04 PM   #117
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,922

Rep: Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040
I managed to get a nice reduction in runtime on my circle program when finding all results, by reworking it a bit this weekend.

Also, it should now only show unique results: no mirrors.

Here's some timings on my latest WIP version:
Code:
$ time ./circle -va 40 |grep results
64 results returned.

real	0m0.814s
user	0m0.817s
sys	0m0.000s
$ time ./circle -va 41 |grep results
464 results returned.

real	0m2.531s
user	0m2.532s
sys	0m0.004s
$ time ./circle -va 42 |grep results
1062 results returned.

real	0m7.798s
user	0m7.800s
sys	0m0.001s
$ time ./circle -va 43 |grep results
4337 results returned.

real	0m24.286s
user	0m24.287s
sys	0m0.002s
$ time ./circle -va 44 |grep results
10091 results returned.

real	1m9.851s
user	1m9.853s
sys	0m0.003s
$
For those of you running your own code, do those "results returned" numbers agree with what you get?

For comparison, this is my old 2.5 version:
Code:
$ time ./circle -va 44 |grep results
20182 results returned.

real	2m41.978s
user	2m41.984s
sys	0m0.004s
$
I've still got a bit more work to do on the code: needs a clean-up after the reworking, and I have some other ideas I want to try.

This has been a really interesting problem to play with.

P.S. I don't have enough experience with python to contribute on that thread of the discussion, but I'm still watching with interest.

Last edited by GazL; 05-09-2021 at 06:25 PM.
 
1 members found this post helpful.
Old 05-10-2021, 12:45 PM   #118
dogpatch
Member
 
Registered: Nov 2005
Location: Central America
Distribution: Mepis, Android
Posts: 490

Original Poster
Blog Entries: 4

Rep: Reputation: 238Reputation: 238Reputation: 238
Quote:
Originally Posted by GazL View Post
Code:
$ time ./circle -va 40 |grep results
64 results returned.

real	0m0.814s
user	0m0.817s
sys	0m0.000s
$ time ./circle -va 41 |grep results
464 results returned.

real	0m2.531s
user	0m2.532s
sys	0m0.004s
$ time ./circle -va 42 |grep results
1062 results returned.

real	0m7.798s
user	0m7.800s
sys	0m0.001s
For those of you running your own code, do those "results returned" numbers agree with what you get?
Yes, i get the same counts 64 464 1062 for ring sizes 40 41 42. Didn't try -a all rings beyond 42, but assume would agree there as well. (These are numbers without mirrors, or duplicates, right?)
 
1 members found this post helpful.
Old 05-10-2021, 01:05 PM   #119
GazL
LQ Veteran
 
Registered: May 2008
Posts: 6,922

Rep: Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040Reputation: 5040
Quote:
Originally Posted by dogpatch View Post
Yes, i get the same counts 64 464 1062 for ring sizes 40 41 42. Didn't try -a all rings beyond 42, but assume would agree there as well. (These are numbers without mirrors, or duplicates, right?)
Thanks for the confirmation.

Yes, these totals exclude mirrored results, which the latest version of my program no longer finds. Also, I don't believe that it was possible for my code to return a non-mirrored duplicate result, even before I changed it.

I'll share the new version once I've had time to clean it up a little.
 
2 members found this post helpful.
Old 05-10-2021, 01:14 PM   #120
dogpatch
Member
 
Registered: Nov 2005
Location: Central America
Distribution: Mepis, Android
Posts: 490

Original Poster
Blog Entries: 4

Rep: Reputation: 238Reputation: 238Reputation: 238
I was all set to upload my latest code, but LQ's new security feature won't let me. May try another way to make it available. Meanwhile, here's my progress report:

A couple performance improvements:
Rather than try various strategems for ordering the array of pairs, am now using a single multi-layered approach. Ordering the pairs in reverse order, favoring larger values first, is generally more efficient because larger numbers tend to have fewer associated pairs. It makes even more sense to favor those target numbers that occur less frequently, which is not alwys the same thing. Using the target frequency method gives me fast results in most ring size / seed cases. Most, but not all.

Quote:
Originally Posted by GazL View Post
. . .I don't really see the point in using an array of pre-determined seed values as one may as well have a array of full results. . .
hmmm. . . an interesting thought. Well, as noted above, i've abandoned the kludgy array of pre-set seed values. But if the program were to quickly pre-set an entire valid array. . .

The target frequency method gives fast results in so many ring size / seed cases that for pretty much all ring sizes there are seed values for which it can give a very fast result. (Tested thoroughly for ring sizes up to 400, spot checked up to 1000.) Using this confidence, my program now solves for a given ring size / seed by trying several seeds in a quick loop until it gets a valid result. The sequence of pairs thus generated is then used to optimize the global pairs array so that the 'real' logic can generate the first valid ring for the 'real' seed with minimal recursions.

This, of course, is only beneficial for the first generated ring; it does not appreciably affect the speed to generate all rings for a given size.
 
1 members found this post helpful.
  


Reply

Tags
circle, perfect square, programming challenge, ring, ring data type



Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is Off
HTML code is Off



Similar Threads
Thread Thread Starter Forum Replies Last Post
damnit!!! i cant see what im typing.. fonts are squares iLLuSionZ Linux - Newbie 22 11-18-2003 03:36 PM
characters as squares in mozilla icyfire Linux - Software 4 09-18-2003 04:22 PM
white squares appear on screen saver? bongo.hickup Linux - Hardware 0 06-13-2003 03:51 AM

LinuxQuestions.org > Forums > Non-*NIX Forums > Programming

All times are GMT -5. The time now is 04:53 AM.

Main Menu
Advertisement
My LQ
Write for LQ
LinuxQuestions.org is looking for people interested in writing Editorials, Articles, Reviews, and more. If you'd like to contribute content, let us know.
Main Menu
Syndicate
RSS1  Latest Threads
RSS1  LQ News
Twitter: @linuxquestions
Open Source Consulting | Domain Registration