More N-Queens Puzzle Results by James Pate Williams, Jr.

N-Queens Tests				
8 Queens	Microseconds		50 Samples	
Algorithm	Minimum	Average	Maximum	Std Dev
Arc-Consistency 3	1846	4148	14220	2294
Arc-Consistency 4	3850	8987	27903	5022
Backjump	175	1057	3873	883
Backmark	198	737	3323	596
Backtracking	171	789	3132	652
Hill-Climber	1991	2097	2913	157
				
9 Queens	Microseconds		50 Samples	
Algorithm	Minimum	Average	Maximum	Std Dev
Arc-Consistency 3	2628	5785	15748	3202
Arc-Consistency 4	7492	18057	56587	11114
Backjump	208	1410	6184	1325
Backmark	214	866	4468	880
Backtracking	210	1269	6156	1186
Hill-Climber	1309	1398	1953	102
				
10 Queens	Microseconds		50 Samples	
Algorithm	Minimum	Average	Maximum	Std Dev
Arc-Consistency 3	3820	10680	30708	6963
Arc-Consistency 4	12624	46645	148221	31212
Backjump	248	3970	16736	3805
Backmark	283	1504	7613	1433
Backtracking	227	2857	12882	3022
Hill-Climber	24575	25608	33563	2037

My hill-climber results are pretty disappointing. I have gotten some decent runs of back-marking for n = 50 and n = 60 queens. I show the n = 60 queens experiment below.

==Menu==
**Instance Submenu**
1 Single Instance Test
2 Multiple Instances Test
3 Exit
Enter an option '1' to '3': 2
**Algorithm Submenu**
1 Arc-Consistency 3
2 Arc-Consistency 4
3 Backjump
4 Backmark
5 Chronological Bactracking
6 Evolutionary Hill-Climber
7 Exit
Enter an algorithm ('1' to '7'): 4
Enter the number of queens for option '2': 60
Enter the number of samples to be analyzed: 10
Minimum Runtime in Microseconds: 4819
Average Runtime in Microseconds: 10419330
Maximum Runtime in Micorseconds: 45439444
Standard Sample Deviation: 16665688

Notice the misspelling of microseconds.

Six C++ Algorithms to Solve the N-Queens Puzzle (Problem) by James Pate Williams, Jr.

Most of the six algorithms were developed back in the years 1999 to 2001 when I was a Master of Software Engineering and Doctoral candidate at Auburn University. Professor Gerry V. Dozier was my Master research advisor. The algorithms developed were in alphabetical order:

  • Arc-Consistency 3
  • Arc-Consistency 4
  • Backjump
  • Backmark
  • Chronological Backtracking
  • Evolutionary Hill-Climber (My algorithm)

The first five algorithms were stated in pseudocode in the excellent treatise Foundations of Constraint Satisfaction by Edward Tsang. Below are six runs of my test C++ application (Win32 Console App) using 8 queens and 50 samples.

The Graph Coloring Problem by James Pate Williams, Jr.

The Graph Coloring Problem and N-Queens Puzzle are both NP-Complete constraint satisfaction problems. There does not exist polynomial time algorithms to solve large instances of either problem. The Graph Coloring Problem is given a graph of n-vertices and m-edges find the minimum number of colors such that no two connected vertices are of the same color. The N-Queens Puzzle is given a n-by-n chessboard and n-queens place the queens such that no two queens are attacking one another. A queen in chess can move any number of squares diagonally, horizontally, or vertically.

As a Master of Software Engineering graduate student in the era 1998 to 2000, I majored in the subdiscipline of artificial intelligence known as constraint satisfaction. I developed evolutionary hill-climbing algorithms to solve constraint satisfaction problems. I specifically solved instances of the Graph Coloring Problem and the N-Queens Puzzle. I compared my algorithms with other researchers’ algorithms. One of the algorithms tested was Makato Yokoo’s Weak-Commitment Search Algorithm (WCSA). Yakoo wrote a very nice book named “Distributed Constraint Satisfaction” which has excellent flow-charts of several algorithms including the Min-Conflicts Search Algorithm (MCSA) and his WCSA. I implemented both algorithms in the C++ and Java computer languages.

I rewrote the preceding two algorithms in the C# computer language sometime in the epoch 2008 – 2009. Below are three instances of the Graph Coloring Problem with four, eight, and twelve vertices solved by the Yakoo’s WCSA.

Guitar Chord and Scale Calculator by James Pate Williams, Jr. (c) 2003-2009 Now 2023

In the early 2000s I bought a hardware SNARLING DOGS GUITAR CHORD Computer. In 2003 I decided to emulate the hardware device in Java and later in 2009 C#. I have more work to do such as adding minor, augmented, diminished, sixth, seventh, and ninth chords. I require more scales such as the seventh and ninth scales.

Third Blast from the Past 1999 to 2007 Mercator Map Projection Using the Open Graphics Library by James Pate Williams, Jr.

I took an advanced graphics class in the spring quarter of 1999 at Auburn University under Professor Kai Chang. We used the Open Graphics Library in the languages C/C++. I later modified my class project into a Mercator map projection application. I have attached some screen shots from the Mercator projection application.

Blast from the Past 1997 Image of a Matrix by James Pate Williams, Jr.

/*
  Author:  Pate Williams (c) 1997

  "Algorithm 2.3.2 (Image of a Matrix). Given an
  m by n matrix M = (m[i][i]) with 1 <= i <= m and
  1 <= j <= n having coefficients in the field K,
  this algorithm outputs a basis of the image of
  M, i. e. vector space spanned by the columns of
  M." -Henri Cohen- See "A Course in Computational
  Algebraic Number Theory" by Henri Cohen pages
  58-59. We specialize the code to the field Zp.
*/

#include <stdio.h>
#include <stdlib.h>

long** create_matrix(long m, long n)
{
    long i, ** matrix = (long**)calloc(m, sizeof(long*));

    if (!matrix) {
        fprintf(stderr, "fatal error\ninsufficient memory\n");
        fprintf(stderr, "from create_matrix\n");
        exit(1);
    }
    for (i = 0; i < m; i++) {
        matrix[i] = (long*)calloc(n, sizeof(long));
        if (!matrix[i]) {
            fprintf(stderr, "fatal error\ninsufficient memory\n");
            fprintf(stderr, "from create_matrix\n");
            exit(1);
        }
    }
    return matrix;
}

void delete_matrix(long m, long** matrix)
{
    long i;

    for (i = 0; i < m; i++) free(matrix[i]);
    free(matrix);
}

void Euclid_extended(long a, long b, long* u,
    long* v, long* d)
{
    long q, t1, t3, v1, v3;

    *u = 1, * d = a;
    if (b == 0) {
        *v = 0;
        return;
    }
    v1 = 0, v3 = b;
#ifdef DEBUG
    printf("----------------------------------\n");
    printf(" q    t3   *u   *d   t1   v1   v3\n");
    printf("----------------------------------\n");
#endif
    while (v3 != 0) {
        q = *d / v3;
        t3 = *d - q * v3;
        t1 = *u - q * v1;
        *u = v1, * d = v3;
#ifdef DEBUG
        printf("%4ld %4ld %4ld ", q, t3, *u);
        printf("%4ld %4ld %4ld %4ld\n", *d, t1, v1, v3);
#endif
        v1 = t1, v3 = t3;
    }
    *v = (*d - a * *u) / b;
#ifdef DEBUG
    printf("----------------------------------\n");
#endif
}

long inv(long number, long modulus)
{
    long d, u, v;

    Euclid_extended(number, modulus, &u, &v, &d);
    if (d == 1) return u;
    return 0;
}

void image(long m, long n, long p,
    long** M, long** X, long* r)
{
    int found;
    long D, i, j, k, s;
    long* c = (long*)calloc(m, sizeof(long));
    long* d = (long*)calloc(n, sizeof(long));
    long** N = create_matrix(m, n);

    if (!c || !d) {
        fprintf(stderr, "fatal error\ninsufficient memory\n");
        fprintf(stderr, "from kernel\n");
        exit(1);
    }
    for (i = 0; i < m; i++) {
        c[i] = -1;
        for (j = 0; j < n; j++) N[i][j] = M[i][j];
    }
    *r = 0;
    for (k = 0; k < n; k++) {
        found = 0, j = 0;
        while (!found && j < m) {
            found = M[j][k] != 0 && c[j] == -1;
            if (!found) j++;
        }
        if (found) {
            D = p - inv(M[j][k], p);
            M[j][k] = p - 1;
            for (s = k + 1; s < n; s++)
                M[j][s] = (D * M[j][s]) % p;
            for (i = 0; i < m; i++) {
                if (i != j) {
                    D = M[i][k];
                    M[i][k] = 0;
                    for (s = k + 1; s < n; s++) {
                        M[i][s] = (M[i][s] + D * M[j][s]) % p;
                        if (M[i][s] < 0) M[i][s] += p;
                    }
                }
            }
            c[j] = k;
            d[k] = j;
        }
        else {
            *r = *r + 1;
            d[k] = -1;
        }
    }
    for (j = 0; j < m; j++) {
        if (c[j] != -1) {
            for (i = 0; i < n; i++) {
                if (i < m) X[i][j] = N[i][c[j]];
                else X[i][j] = 0;
            }
        }
    }
    delete_matrix(m, N);
    free(c);
    free(d);
}

void print_matrix(long m, long n, long** a)
{
    long i, j;

    for (i = 0; i < m; i++) {
        for (j = 0; j < n; j++)
            printf("%2ld ", a[i][j]);
        printf("\n");
    }
}

int main(void)
{
    long i, j, m = 8, n = 8, p = 13, r;
    long a[8][8] = { {0,  0,  0,  0,  0,  0,  0,  0},
                    {2,  0,  7, 11, 10, 12,  5, 11},
                    {3,  6,  3,  3,  0,  4,  7,  2},
                    {4,  3,  6,  4,  1,  6,  2,  3},
                    {2, 11,  8,  8,  2,  1,  3, 11},
                    {6, 11,  8,  6,  2,  6, 10,  9},
                    {5, 11,  7, 10,  0, 11,  6, 12},
                    {3,  3, 12,  5,  0, 11,  9, 11} };
    long** M = create_matrix(m, n);
    long** X = create_matrix(n, n);

    for (i = 0; i < m; i++)
        for (j = 0; j < n; j++)
            M[i][j] = a[j][i];
    printf("the original matrix is as follows:\n");
    print_matrix(m, n, M);
    image(m, n, p, M, X, &r);
    printf("the image of the matrix is as follows:\n");
    print_matrix(n, n - r, X);
    printf("the rank of the matrix is: %ld\n", n - r);
    delete_matrix(m, M);
    delete_matrix(n, X);
    return 0;
}

First Blast from the Past (1997) Computing the Inverse Image of a Matrix by James Pate Williams, Jr.

/*
  Author:  Pate Williams (c) 1997

  "Algorithm 2.3.5 (Inverse Image Matrix). Let M be
  an m by n matrix and V be an m by r matrix, where
  n <= m. This algorithm either outputs a message
  saying that some column vector of V is not in the
  image of M, or outputs an n by r matrix X such
  that V = MX." -Henri Cohen- See "A Course in Com-
  putational Algebraic Number Theory" by Henri
  Cohen pages 60-61. We specialize to the field Q.
*/

#include <stdio.h>
#include <stdlib.h>

double** create_matrix(long m, long n)
{
    double** matrix = (double**)calloc(m, sizeof(double*));
    long i;

    if (!matrix) {
        fprintf(stderr, "fatal error\ninsufficient memory\n");
        fprintf(stderr, "from create_matrix\n");
        exit(1);
    }
    for (i = 0; i < m; i++) {
        matrix[i] = (double*)calloc(n, sizeof(double));
        if (!matrix[i]) {
            fprintf(stderr, "fatal error\ninsufficient memory\n");
            fprintf(stderr, "from create_matrix\n");
            exit(1);
        }
    }
    return matrix;
}

void delete_matrix(long m, double** matrix)
{
    long i;

    for (i = 0; i < m; i++) free(matrix[i]);
    free(matrix);
}

void inverse_image_matrix(long m, long n, long r,
    double** M, double** V,
    double** X)
{
    double ck, d, sum, t;
    double** B = create_matrix(m, r);
    int found;
    long i, j, k, l;

    for (i = 0; i < m; i++)
        for (j = 0; j < r; j++)
            B[i][j] = V[i][j];
    for (j = 0; j < n; j++) {
        found = 0, i = j;
        while (!found && i < m) {
            found = M[i][j] != 0;
            if (!found) i++;
        }
        if (!found) {
            fprintf(stderr, "fatal error\nnot linearly independent\n");
            fprintf(stderr, "from inverse_image_matrix\n");
            exit(1);
        }
        if (i > j) {
            for (l = 0; l < n; l++)
                t = M[i][l], M[i][l] = M[j][l], M[j][l] = t;
            for (l = 0; l < r; l++)
                t = B[i][l], B[i][l] = B[j][l], B[j][l] = t;
        }
        d = 1.0 / M[j][j];
        for (k = j + 1; k < m; k++) {
            ck = d * M[k][j];
            for (l = j + 1; l < n; l++)
                M[k][l] -= ck * M[j][l];
            for (l = 0; l < r; l++)
                B[k][l] -= ck * B[j][l];
        }
    }
    for (i = n - 1; i >= 0; i--) {
        for (k = 0; k < r; k++) {
            sum = 0;
            for (j = i + 1; j < n; j++)
                sum += M[i][j] * X[j][k];
            X[i][k] = (B[i][k] - sum) / M[i][i];
        }
    }
    for (k = n + 1; k < m; k++) {
        for (j = 0; j < r; j++) {
            sum = 0;
            for (i = 0; i < n; i++)
                sum += M[k][i] * X[i][j];
            if (sum != B[k][j]) {
                fprintf(stderr, "fatal error\ncolumn not in image\n");
                fprintf(stderr, "from inverse_image_matrix\n");
                exit(1);
            }
        }
    }
    delete_matrix(m, B);
}

void matrix_multiply(long m, long n, long r,
    double** a, double** b,
    double** c)
    /* c = a * b */
{
    double sum;
    long i, j, k;

    for (i = 0; i < m; i++) {
        for (j = 0; j < r; j++) {
            sum = 0.0;
            for (k = 0; k < n; k++)
                sum += a[i][k] * b[k][j];
            c[i][j] = sum;
        }
    }
}

void print_matrix(long m, long n, double** a)
{
    long i, j;

    for (i = 0; i < m; i++) {
        for (j = 0; j < n; j++)
            printf("%+10.6lf ", a[i][j]);
        printf("\n");
    }
}

int main(void)
{
    long i, j, m = 4, n = 4, r = 4;
    double** c = create_matrix(m, n);
    double** M = create_matrix(m, n);
    double** V = create_matrix(m, r);
    double** X = create_matrix(n, r);

    for (i = 0; i < m; i++) {
        c[i][i] = M[i][i] = 2.0;
        if (i > 0)
            c[i][i - 1] = M[i][i - 1] = -1.0;
        if (i < m - 1)
            c[i][i + 1] = M[i][i + 1] = -1.0;
    }
    for (i = 0; i < m; i++)
        for (j = 0; j < r; j++)
            V[i][j] = i + j + 1;
    printf("M is as follows:\n");
    print_matrix(m, n, M);
    printf("V is as follows:\n");
    print_matrix(m, r, V);
    inverse_image_matrix(m, n, r, M, V, X);
    printf("X is as follows:\n");
    print_matrix(n, r, X);
    matrix_multiply(m, n, r, c, X, M);
    printf("MX is as follows:\n");
    print_matrix(m, r, M);
    delete_matrix(m, c);
    delete_matrix(m, M);
    delete_matrix(m, V);
    delete_matrix(n, X);
    return 0;
}

Path Genetic Algorithm to Solve the Traveling Salesperson Problem by James Pate Williams, Jr.

I have been interested in solving the Traveling Salesperson problem (TSP) since the early 2000’s. I have used several evolutionary techniques. This blog will cover my implementation of a path genetic algorithm created by me in 2017. Of course, the world’s best TSP solver is the University of Waterloo Conrcorde:

https://www.math.uwaterloo.ca/tsp/concorde.html

which now uses a linear programming solver. I used the programming language C# in my implementations.

The previous graph reminds me of John Travolta dancing to the Bee Gees hit song “Staying Alive”.

The previous data for a 127-city tour is not optimal.

The left length is 123,004 and the right length is 121,697. I don’t know if the right length is optimal. The tour in the literature is bier-127.

Multiple Precision Arithmetic in C++ Implemented by James Pate Williams, Jr.

I have developed a long integer package in C++ using H. T. Lau’s “A Numerical Library in C for Scientists and Engineers”. That library is based on the NUMAL Numerical Algorithms in Algol library. I use 32-bit integers (long) as the basis of the LongInteger type. The base (radix) is 10000 which is the largest base using 32-bit integers. As a test of the library, I use Pollard’s original rho factorization method. I utilized the Alfred J. Menezes et al “Handbook of Applied Cryptography” Miller-Rabin algorithm and ANSI X9.17 pseudo random number generator with Triple-DES as the encryption algorithm. I translated Hans Riesel Pascal code for Euclid’s algorithm and the power modulus technique. I don’t use dynamic long integers a la Hans Riesel’s Pascal multiple precision library. The single precision is 32 32-bit longs and multiple precision 64 32-bit longs.

Here is a typical factorization:

Number to be factored, N = 3 1234 5678 9012
Factor = 3
Is Factor prime? 1
Factor = 4
Is Factor prime? 0
Factor = 102 8806 5751
Is Factor prime? 1
Function Evaluations = 6
Number to be factored, N = 0

The first number of N is the number of base 10000 digits. I verified that 10288065751 was prime using Miller-Rabin and the table found online below:

http://compoasso.free.fr/primelistweb/page/prime/liste_online_en.php