Chain of thought prompting
The technique of Chain-of-Thought (CoT) prompting evaluates the ability of an AI model to break complex problems into a series of logical steps. The essay embedded in this post was applied to generative artificial intelligence (GenAI) models to test their ability to process geometric numerical analysis. Their outputs offer insight into breakthroughs at the forefront AI chatbot development.
Critical Thinking
The square of the hypotenuse (156.25) is actually greater than the sum of the squares on the sides (241.25) by 85.
— Meta Llama 3.1 8B-Instruct on scaled down hardware. (156.25 is actually less than 241.25)
CoT prompting was applied to two open-source large language models (LLMs) running on a single GPU isolated from the Internet. Under these conditions smaller models produced output that is clearly wrong when examined with critical thinking.
One model asserted that the essay was not entirely correct because 156.25 is greater than 241.25. A second model failed to process geometric logic and failed at number substitution.
The prompt was also applied to large-scale GenAI models running on cloud servers. These evaluated the essay correctly. The outputs of four models are shown as screenshots in the appendices to this post.
Not tested was the recently announced OpenAI o1. This new series of models was designed to spend more time processing before they respond, in order to solve harder problems in science, coding and math. Such as the essay below.
(As an aside to my regular readers, the AiJunzi platform was upgraded to Microsoft Phi 3.5 since my last post.)
Discerning Pythagorean and Euclidean Logic
Pythagoras and Euclid are two of the most influential figures in the history of mathematics. Pythagoras is best known for the Pythagorean Theorem, which states that the square of the hypotenuse on a right triangle is equal to the sum of the squares of the other two sides.
a² + b² = c²
Euclid wrote a comprehensive description of geometry. The proofs in Euclid’s “Elements” are known for their logical rigor and their systematic approach. In this essay I propose that the Euclidean method can be explained using numerical analysis.
Numerical analysis concisely verifies mathematical propositions
One of Euclid’s proofs is a statement of the Pythagorean Theorem. However, reading through Proposition 47, we find that it relies on earlier propositions, which themselves rely on earlier postulates. This can make it difficult to understand the underlying logical principles. There’s a lot to unpack here. We can use the Pythagorean Theorem and numeric values to provide an intuitive understanding of one of Euclid’s propositions.
Acute triangles and squared sides by the numbers
Let’s look at Book 2, Proposition 13 using numerical analysis. Referring to Figure 1 (top) and part of the proof (below) we can simply state that (AB)² = (BD)² + (DA)².
But the square on AB is equal to the squares on BD, DA, for the angle at D is right.
— Elements, Book 2 Proposition 13
To substitute numeric values we can use Pythagorean triples, which are sets of three positive integers that satisfy the equation (Table 1). To make this work, we’ll divide one of those sets by two. This way we get numbers that are, if not whole, easy to work with.
| a | b | c |
| 5 | 12 | 13 |
| 7 | 24 | 25 |
| 3.5 | 12 | 12.5 |
Finally, we add the sets of numbers from Table 1 to Figure 2, and conclude with a re-statement of Book 2, Proposition 13.
From logical to numeric reasoning
In acute-angled triangles the square on the side subtending the acute angle is less than the squares on the sides containing the acute angle by twice the rectangle contained by one of the sides about the acute angle, namely that on which the perpendicular falls, and the straight line cut off within by the perpendicular towards the acute angle.
— Elements, Book 2 Proposition 13
Following along with Euclid’s argument, we have an acute triangle ABC, and have drawn a perpendicular from point A to point D. (Figure 2). Observe that the perpendicular is a common side on two right triangles inside ABC, and that the value 12 is assigned to it. (We have factored the table of Pythagorean triples to make it so.)
The proposition says that the square of side (AC) is less than the squares on the sides (CB) and (BA) by twice the value contained by (CB), (BD). Or
(12.5)² is less than (8.5)² plus (13)² by 2 times (8.5 x 5)
Re-arranging terms
241.25 – 156.25 = 85
Therefore, in this acute triangle the square of the hypotenuse subtending angle B (156.25) is less than squares on the sides (241.25) by 2 x (8.5 x 5).
Q.E.D.
Appendix A: Meta Llama 3.1 B-Instruct (open-source)
“The square of the hypotenuse (156.25) is actually greater than the sum of the squares on the sides (241.25) by 85”. False.
Appendix B: Microsoft Phi 3.5 mini-instruct (open-source)
“… the sum of the squares on sides BD (8.5) and CB (13) by a factor of 2 times the product of the perpendicular (8.5)…” Orthographic errors. Incorrect numeric and label substitution.
Appendix C: OpenAI ChatGPT (online)
No issues
Appendix D: Google Gemini (online)
No issues
