Last gasp stylometry from the Interpreter

[I have mixed feelings about even responding to Brother Roper’s stylometry article, found below, which was published in the Interpreter. My every effort in these fields (Church history and Book of Mormon geography) is to seek consensus about the facts and the most rational conclusions based on those facts, all with the goal of building the faith of those who believe in the historicity of the Book of Mormon and the credibility of its first witnesses.

So on one hand, I’ve been inclined to ignore this article, partly because of its tone but also because of its misleading content. I have difficulty believing anyone takes this study seriously, apart from those whose interest consists of confirming their biases, such as the editors of the Interpreter and other staunch Mesoamerican proponents. The overall complaint in the article is humorous. Last January, I specifically requested the underlying data and sought to collaborate on a full and robust stylometry analysis; after initially agreeing, Brother Roper refused to proceed. Consequently, I was left with only what was reported in the original Maxwell Institute paper, found here. Now they criticize me for not having analyzed the data they refused to provide me! 

On the other hand, there may be some interested readers who could be misled by the article if no one points out the fallacies. And we can be sure the Interpreter will never publish that.

Hence, this response.

My overall conclusion: this last-gasp effort to preserve the Mesoamerican geography is even worse than I expected. As before, the authors provide no information about their data, their software, or their parameters; their results cannot be replicated. In fact, they provide even less data about their new analysis than they did about their original one.

The article makes a hash of what was, in their original article, at least coherent. Anyone who reads to the bitter end will see that the criteria they apply to Winchester, Phelps, Wm Smith and others flatly contradict the criteria they apply to Joseph Smith, Woodruff, and Taylor. The authors utterly fail to address the many historical errors they committed in their first article, but add more here.

Bottom line: people who want to believe Joseph Smith wrote the ridiculous, factually false articles in the Times and Seasons can use this article to confirm their bias, but no objective reader will come away persuaded by the stylometry arguments.

Note: Throughout this article I refer to Brother Roper. This is not a slight to Brother Fields or Brother Bassist, but Brother Roper is listed first and for simplicity I can only refer to one author. I’ll assume that each of the three contributed equally to every part of the article. Perhaps a stylometry analysis would prove otherwise…except I don’t think stylometry is any good for co-authored and edited material.

J]

Abstract: This article is the third in a series of three articles responding to the recent assertion by Jonathan Neville that Benjamin Winchester was the anonymous author of three unsigned editorials published in Nauvoo in 1842 in the Times and Seasons. The topic of the unsigned editorials was the possible relationship of archeological discoveries in Central America to places described in the Book of Mormon narrative. The first article shows that, contrary to Neville’s claims, Winchester was not a proponent of a Mesoamerican setting for the Book of Mormon, but rather a hemispheric one.

[I’ve shown that Brother Roper’s assertions in that review were specious, but I was grateful to Brother Roper for helping to make my argument. In fact, I incorporated some of his observations into my second edition because they corroborate what I wrote in the first edition.]

Since this was a view commonly held by early Mormons, his ideas did not warrant any anonymity for their dissemination.

[The 15 Sep and 1 Oct T&S articles are unique; no one before or after put Zarahemla in Quirigua or claimed “we read” in the Book of Mormon that Lehi landed in Central America. The “commonly held” view was expressed by Orson Pratt; i.e., that Zarahemla was in South America. Although Joseph explicitly rejected that view, it resurfaced in 1879 when Pratt added footnotes to the Book of Mormon, which John Taylor presumably approved.]

The second article shows that, also contrary to Neville’s claims, Joseph Smith was not opposed to considering Central American geographic parallels to the Book of Mormon. The Prophet even seemed to find such possibilities interesting and supportive of the Book of Mormon.

[There is zero direct evidence to support this assertion, and Brother Roper’s assertions are based on a series of unsubstantiated assumptions that contradict the historical evidence.]

This third article shows that despite Neville’s circumstantial speculations, the historical and stylometric evidence is overwhelmingly against Winchester as the author of the Central America editorials.

[The historical evidence shows Winchester, Phelps, and Wm. Smith were the main contributors to the Times and Seasons after May 1842, through October 1, 1842, and no legitimate stylometric evidence contradicts this.]

In The Lost City of Zarahemla from Iowa to Guatemala — and Back Again, novelist Jonathan Neville tries to discredit what he calls the “limited Mesoamerican geography” of Book of Mormon events.¹ To do so he argues that Benjamin Winchester — an early Mormon missionary, writer and eventual apostate — was the anonymous author of three unsigned editorials published in the Times and Seasons on correspondences between the discoveries in Central America by Stephens [Page 14]and Catherwood and the Book of Mormon.^2 3 The three editorials were published during Joseph Smith’s editorship of the Times and Seasons from March to November of 1842.

Neville claims that Joseph Smith was opposed to drawing Mesoamerican connections with the Book of Mormon account and felt that they posed a threat to his prophetic authority. In order to explain how the three editorials came to be published, Neville invents an elaborate tale of subterfuge and conspiracy masterminded by Winchester.

[It’s strange that Brother Roper calls me a novelist when everything I wrote in the Zarahemla book is fully documented, while Brother Roper is forced to invent all kinds of activities Joseph supposedly engaged in that are nowhere supported in the historical record.]

Neville also includes a pseudo-stylometric “analysis” in an attempt to support his speculations. Neville is the author of at least twelve self-published novels. He is an attorney by training. He is not a historian, statistician, or stylometrician.

[I readily admit I’m not a professional historian, statistician, or stylometrician, but neither is Brother Roper. A good sign of a losing argument is over-reliance on “expert testimony” when the expertise of the experts has not been established and their work is so easily impeached. That said, I welcome an open, professional analysis if anyone wants to try one.

As I point out in the book, I focused on the historical context and the content of the articles. I explained in the book that I’m dubious of stylometry generally, and especially “black-box” stylometry that doesn’t disclose the underlying data, the software used, the specific assumptions made, or the specific parameters applied.

The limited stylometry work previously published by these authors has been criticized by others, but mostly it is ignored. As it should be.

That this paper and the other two reviews were published in the Interpreter means they were not peer-reviewed and are intended to advance a particular viewpoint. Those who seek merely to confirm their beliefs that Joseph Smith didn't know much about the Book of Mormon, speculated about its geography, and wrote anonymous articles need not read my response. The Interpreter surely won’t publish it. People ultimately choose what they want to believe, often in spite of the evidence. If that were not the case, the Mesoamerican theory never would have gained the prominence it once had, and its demise would have occurred long ago. The positions expressed in Brother Roper’s article, in my view, contradict the historical evidence and any logical, rational inferences that can be drawn therefrom.]

This paper is the third in a series of three articles that address Neville’s assertions. In the first article Matthew Roper showed that what Neville characterized as the “limited Mesoamerican geography” of the Book of Mormon was actually the traditional hemispheric view, which assumed that North and South America were the lands described in the Book of Mormon and that Central America was the “narrow neck of land” referred to in the account.⁴ Winchester’s writings merely reflected that commonly held perspective, which was never challenged during Joseph Smith’s lifetime. Neville claims that Winchester and possibly a co-conspirator had to conceal their identities in order to get their ideas published. But, since the idea that the “narrow neck of land” was in Central America was a widely held view, Mormons did not think of it as controversial, and the motive for a secret publication scheme evaporates. Winchester’s writings did not present anything especially new or controversial; thus there was no need for subterfuge and conspiracy to publish them.

[Brother Roper ignores two key facts: 1) people writing in the Times and Seasons were using pseudonyms, and articles were published anonymously. Why? This point will resurface shortly. More important here, if these Central American articles were not new or controversial, why has Brother Roper worked so hard to link Joseph Smith to them?]

In the second article Roper discussed the influence of Stephens and Catherwood’s work Incidents of Travel in Central America on early [Page 15]Latter-day Saint readers, including Joseph Smith.⁵ The Prophet embraced with interest and enthusiasm the book’s description of Central American history and ancient ruins, and asserted they corresponded with and supported the Book of Mormon account.

[The historical record contains zero evidence of Joseph Smith’s personal interest in Central America; at most, one can draw inferences from anonymous articles and the Bernhisel note, the author of which is also unknown. One paragraph earlier, Brother Roper claimed that there was no need for anyone to write articles in secret. But if there was no need, then why are these articles anonymous? Why would Joseph Smith write anything anonymously? Of all people, he is the least likely to spend time writing something, only to leave his name off. In both of the very issues these articles were published in—15 Sep and 1 Oct 1842—Joseph had other letters published that emphasized him as their author. Ironically, for those who promote a Mesoamerican geography, it is a good thing that these articles were anonymous, because now they can claim that Joseph wrote them—in spite of the historical evidence to the contrary.]

In this third article we apply statistical and stylometric analyses to examine whether Winchester is a likely author of the three unsigned Central America editorials. We first summarize the results of our previous paper — “Joseph Smith, The Times and Seasons, and Central American Ruins” — regarding the Central America editorials, since Neville used that article to form the foundational premise for his book.⁶ We show that his premise is invalid, and therefore the entire argument put forth in his book is baseless. However, going further to address the specific assertions in his book, we explain “stylometry,” discuss Neville’s pseudo-stylometry, and present the results of appropriate stylometric analyses.⁷ Our results show consistently that Winchester is not a viable candidate author of the Central America editorials, and there is no evidence that he is a better candidate than Joseph Smith.

Summary of “Joseph Smith, the Times and Seasons, and Central American Ruins”

Candidate Authors Used: Joseph Smith, John Taylor and Wilford Woodruff

As we discussed in “Joseph Smith, the Times and Seasons, and Central American Ruins,” there are sound reasons to consider Joseph Smith, John Taylor, and Wilford Woodruff as potential candidate authors of the unsigned material on Central America and the Book of Mormon. All three men were in Nauvoo during the time of publication, they were responsible for the publication of the Times and Seasons, and they were all familiar with Stephens and Catherwood’s work. As Roper shows in the second article of this series, Joseph Smith’s letter to John Bernhisel shows that Joseph Smith shared the enthusiasm of his companions about [Page 16]the correspondences between Central American history, Stephens and Catherwood’s discoveries, and the Book of Mormon.

[Most of the precedent paragraph is fantasy, not fact. Brother Roper’s initial article falsely claimed that John Taylor wrote the Bernhisel letter. There is no evidence that anyone other than Woodruff knew or cared about Stephens, and Woodruff was ill and absent in the months leading up to the 15 Sept publication of the first two editorials. Worse, Brother Roper himself already excluded Woodruff as a candidate. A potential author's physical presence in Nauvoo is irrelevant; most of the material that appears in the Times and Seasons was mailed in. And not only is there no evidence that Taylor and Woodruff were working at the Times and Seasons prior to the 15 September issue, there is strong evidence that they were not doing so.]

On February 19, 1842, after Joseph Smith had taken control of the Times and Seasons, Wilford Woodruff wrote in his journal, “Joseph the Seer is now the Editor of that paper & Elder Taylor assists him in writing while it has fallen to my lot to take charge of the Business part of the establishment.”⁸

[Note that Woodruff’s statement was in February. At that point, I agree Joseph was working as editor and writer; after all, he was publishing the Book of Abraham. His journal discusses his activities at the paper in this time frame. But his activity at the paper soon declined to nothing, both in reality and as reflected in his journal.]

In a later recollection, John Taylor provided some insight into what it was like writing for the Prophet and then having him critique and correct what John Taylor had written. The subject on one occasion had to do with priesthood keys, the judgment, and the Ancient of Days.

In speaking with the Prophet Joseph once on this subject, he traced it from the first down to the last, until he got to the Ancient of Days. He wished me to write something for him on this subject, but I found it a very difficult thing to do. He had to correct me several times. We are told that the “judgment shall sit and the books be opened.” He spoke of the various dispensations and of those holding the keys thereof, and said there would then be a general giving up or accounting for. I wrote that each one holding the keys of the several dispensations would deliver them up to his predecessor, from one to another, until the whole kingdom should be delivered up to the Father, and then God would be “all in all.” Said he, “That is not right.” I wrote it again, and again he said it was not right. It is very difficult to find language suitable to convey the meaning of spiritual things. The idea was that they should deliver up or give an account of their administrations, in their several dispensations, but that they would all retain their several positions and Priesthood. The Bible and Doctrine and Covenants speak about certain books which should be opened; and another book would be opened, called the Book of Life, and out of the things written in these books would men be judged at the last day.⁹

[John Taylor said this in 1876 at a funeral service in Salt Lake City. Nothing in this passage suggest or implies Taylor was referring to the 1842 Times and Seasons while Joseph was the nominal editor. John Taylor knew and worked with Joseph both before and after that time frame. The subject matter of the editorials—placing the Book of Mormon in Mesoamerica—is hardly the sort of “spiritual things” for which it is “very difficult to find language suitable to convey.” Joseph Smith’s writings, including the Doctrine and Covenants, have a few things to say about Priesthood, keys, and judgment. By contrast, not a single one of his writings mentions Central America or Stephens.]

John Taylor’s account suggests that in working with John Taylor, Joseph Smith would sometimes explain in his own language what he wanted written. Then John Taylor would write, after which the Prophet [Page 17]would critique and correct, sometimes repeatedly, what John Taylor wrote if needed. While we do not know if the same process was followed in all the writing done under his direction, it does suggest that Joseph Smith could be very involved in the process, particularly if he considered it a matter of significant doctrinal importance.

[This is a straw man argument. No one suggests Joseph wouldn’t critique others in their writing. That’s a far cry from claiming Joseph wrote (or dictated) unsigned articles in the Times and Seasons.]

The Prophet also placed Incidents of Travel in Central America, Chiapas, and Yucatan in the Nauvoo Library and Literary Institute — a strange decision if he disapproved of the use of the books by John Taylor, Wilford Woodruff, and others.¹⁰

[Brother Roper’s argument here is important to assess because it is typical of his approach in his other work. Here, in one sentence, Brother Roper offers a trifecta of poorly reasoned arguments.

First, Brother Roper adopts the Michael D. Quinn argument from Early Mormonism and the Magic World View; i.e., Brother Roper, like Quinn, claims that Joseph read every book he owned or had access too. I disagree with both Quinn and Roper on this point. There is little historical evidence Joseph read much other than the scriptures. In 1841-2, the only book his journal shows him reading is the Book of Mormon.

Second, Brother Roper leaves out key facts. Whenever you read something Brother Roper has written, it’s a good idea to check his sources (as Earl Wunderli pointed on with respect to Brother Roper's review of his book). This is one of the reasons I don’t think the Interpreter peer reviewed this piece, except by friends who are pre-disposed to endorse his beliefs regardless—and often in spite of—the evidence. Brother Roper finds just enough evidence to support his preconceptions and then he stops.

Here, Brother Roper points out that Joseph donated the Stephens books to the Nauvoo Library. That's an indisputable historical fact. But Brother Roper doesn’t tell his readers that the Stephens books were just two of many he donated. Along with the Stephens book, here are the other books Joseph Smith donated at the same time to the Nauvoo Library: (see http://mormonhistoricsites.org/wp-content/uploads/2013/04/The-Complete-Record-of-the-Nauvoo-Library-and-Literary-Institute.pdf)

Review of Edwards on the Will

Life of Tecumseh

Whepleys Compend

Scotts Poetical Works in 5 vols

Gillmores Lectures

Merrills Harmony

Epicureo

Krumanachers Works

Catholic Piety

Home Physician

Apochryphal Testament

Bruns’ Travels

Rebel & other Travels 

Browns’ Appeal. gram

Browns English Syntascope

Studies in Poetry & Prose 

Old World & the New – Vol 1st

Voyage & Travels of Ross Perry & others

Bennetts Book Keeping 2 Copies

Incidents of Travel in Yucatan by Stephens 2 Vo

Stephens Travels in Central America 2 Vo

Mosheims Church History 1 Vol 1.50

Times & Seasons 1 2 & 3 Vol also Vol 1 & 2

Dick’s Philosophy

Millenium & other Poems

Beaumonts Experiments

Dictionary of the Holy Bible

Parkers Lectures on Universalism

Sanders Discourse

Metropolitan

Goodrich’s History of the United States

Doddridges Sermons

Catholic Manuel

Whelpleys Compend

Herveys Meditations

Historia de Charles

Rollin 2 Vol

Book of Mormon

By Brother Roper’s logic, Joseph was a well-read librarian--a speed reader, even. By contrast, we do have a detailed record of his reading habits in this time frame. Wilford Woodruff’s journal notes that Joseph started reading the Book of Mormon on 5 December 1841. On 15 January, 1842, Joseph’s journal notes he had reached page 54. That is 54 pages over 41 days, or about 1.3 pages/day. At his rate, it would have taken Joseph nearly two years to read the Stephens books (not to mention all the other books he donated to the Nauvoo library). Brother Roper ignores this actual data in favor of his unsupported assumption that Joseph read the Stephens books in six weeks or less (based on the time between Woodruff's arrival in Nauvoo and the date of the Bernhisel note). And all of this took place without anyone else commenting on Joseph’s sudden interest in Central America or his new—and temporary—speedreading skill.

Furthermore, Brother Roper’s argument requires that Joseph must have not only read all these books, but approved “the use of the books by John Taylor, Wilford Woodruff, and others.” I’d be interested in whatever evidence Roper can produce of Taylor, Woodruff, or anyone else using the Catholic Manuel or Catholic Piety, or of Joseph Smith approving such use.

Bottom line, Joseph's donation of the Stephens books puts them in the same category as Catholic Piety. The donation has zero evidentiary value regarding his supposed enthusiasm for them as proof that the Book of Mormon cities were in Central America.

Now for the third faulty argument in the trifecta. 

 Brother Roper assumes his theory is fact (i.e., that Taylor, Woodruff and others “used” the Stephens book, when the only evidence is that Woodruff read them and an anonymous person used them in unsigned editorials). Then he uses his unsupported theory to form a false argument by turning my argument inside out. Nowhere have I written or suggested that Joseph Smith disapproved of these books. In my view, there is no evidence he cared about them at all, any more than he cared about Catholic Piety. In addition, it is not inconsistent with what Joseph actually said and wrote to consider post-Book of Mormon ruins in Central America as evidence of post-Book of Mormon people who may have included migrants from Lamanite territory. In fact, the Aztalan article in the Times and Seasons makes that very argument. Thus, there is no problem with people citing Stephens as general evidence of the Book of Mormon; the problem is claiming the text says Lehi landed there and that Zarahemla is in Quirigua (or nearby), which are the unique features of the anonymous editorials published on Sept. 15 and Oct. 1.]

John Taylor and Wilford Woodruff were seriously ill during August and September, and both men were essentially bedridden from August 9 until middle to late September. This suggests the chance that someone besides them might have penned the editorials for September 15 and October 1. Neville thinks that Joseph Smith could not have contributed to the content of any of the articles because he was in hiding from his enemies in September 1842 and may not have been able to visit the printing office during that time.

[This is not my argument. Brother Roper borrowed this from somewhere else—another common characteristic of his three reviews, as I’ve noted. My argument is based on simple facts. Joseph wrote a letter on 1 Sept and sent it to the editor of the Times and Seasons, who published it on 15 Sept. He wrote another one on 6 Sept and sent it along. This one was published on 1 Oct. In both cases, the editor publicized Joseph’s authorship of the articles, knowing that such communications were highly valued by his readers. Based on these facts, it is certainly possible that Joseph could have written the 900 words and provided the several thousand words of extracts. I’ve never said otherwise.

It’s not a question of whether he could have, but whether he did. Brother Roper takes the position that while 1) Joseph signed the two letters that became Sections 127 and 128 in the Doctrine and Covenants and 2) sent them to the editor for publication, and 3) the editor introduced them in the newspaper by touting Joseph’s authorship, for some reason Joseph separately 1) wrote the three short comments, 2) attached them to extensive extracts from Stephens, 3) published them himself (or directed the editor to publish them) 4) anonymously. There could not be a more stark disparity between the treatment and content of the two sets of documents.]

None of the editorial portions of the Central America articles for 1842 were long, and they would not have required an inordinate amount of time to write. [This is a remarkable argument. What is “inordinate” under the circumstances? Readers should think about this. The 900 words are not long, granted. But the extracts are carefully selected from the 900 pages of the Stephens books. Joseph supposedly read these books nearly a year earlier. Now, in September, when he’s evading the authorities and seeking assistance from the government and friends, when he’s focusing on the temple to write Sections 127 and 128, when he’s dictating the Book of the Law of the Lord, handling the real estate transactions in Nauvoo, managing the bankruptcy he declared a few months earlier, and so on, he supposedly took the time to find important extracts from the 900 pages of Stephens books and write editorials about them. Just in terms of logistics, this is unlikely. On top of that, he had to instruct his scribes not to mention all his work with Stephens, tell everyone he knew not to mention it, and make sure the editor of the Times and Seasons kept his work anonymous—exactly the opposite of what he did with sections 127 and 128. In Brother Roper’s version of events, Joseph’s involvement with the Stephens books was the best-kept secret in Nauvoo. It was kept more secret than plural marriage.]

The editorials for July 15 (signed “Ed”) and September 15 (unsigned) both [there are actually two in September] mention Stephens and Catherwood’s discoveries and have a similar theme. The September 15 material may have been written in September, but it could have been written just as easily previous to John Taylor’s and Wilford Woodruff’s illnesses in July or early August.

[The editorial in July was primarily a 1,200-word extract from Josiah Priest’s American Antiquities. Benjamin Winchester is the only contributor to the Times and Seasons during 1842 known to have had a copy of Priest’s book.  There is no evidence that anyone in Nauvoo had a copy. Brother Roper not only has Joseph writing anonymous articles, but now he has him extracting books he doesn’t own.

As I’ve pointed out, the articles could have been written by literally anyone with access to the mail. (Parley P. Pratt quoted from American Antiquities in his Voice of Warning, but there’s no evidence he owned a copy of his own, and besides, he was in England in 1842. Of course, he could have mailed in this article like anyone else, but why would he insist on anonymity?) Theoretically, the articles could even be holdovers from deceased persons such as Don Carlos. That’s the point: they are anonymous. Brother Roper has concocted a false, counterfactual requirement that these articles had to have been written by someone living in Nauvoo.]

Neville notes that in Joseph Smith’s journal, “There is no hint that Joseph is reading, writing, or conversing about any topic related to Book of Mormon geography” (p. 130). While true, the way that Neville presents this information is misleading. Indeed, as far as we are aware, there is no hint of any discussion of Book of Mormon geography in anyone’s Nauvoo journal during 1842.

[??? This is precisely my point! Apart from Wilford Woodruff’s journal in September 1841 and the Bernhisel note in November 1841, no one in Nauvoo cared about Book of Mormon geography. Here’s another example. In May 1841, a man from New York presented Joseph Smith with a 20-foot long scroll that had been taken from a rock in South America that purported to depict Lehi crossing the sea and other Book of Mormon events. Few people know about this incident because neither Joseph nor anyone close to him gave it any credence.

Brother Roper here is supporting my argument. The people living in Nauvoo were focused on building the temple, building the city, farming, welcoming new Saints, and sending out missionaries. It was the missionaries in the field, such as Winchester in Philadelphia, who were dealing on a daily basis with the missionary effort, battling with the press and antagonistic ministers, etc. That’s another reason why I think it was someone outside Nauvoo mailing the Priest and Stephens material to the newspaper. And most of the material published in the Times and Seasons was mailed in.]

Thus, the journal’s silence cannot be taken to mean very much, since someone was interested in it, and the Times and Seasons published a handful of articles relating to that subject while Joseph Smith was editor, despite the subject’s absence from anyone’s personal diaries.

[Well. Start with “the journal.” Joseph Smith’s journal was not like everyone else’s. It was kept by scribes; it was kept daily; it was kept in response to a commandment; and it was relatively meticulous. Everything we know he wrote, including the Book of Abraham and the Book of the Law of the Lord, is accounted for in the 1842 journal. To have such major pronouncements as we have in the 900 words (let alone the extracts from American Antiquities), with no mention of them in the journal, is exceptional. Unique, even. Next, we have Woodruff’s journal, probably the most complete apart from Joseph’s, in which he claimed to try to write down everything Joseph said. What did Woodruff record regarding Joseph’s comments about Stephens? Nothing. We have Phelps and Woodruff and others writing letters, mentioning Joseph and his activities, but no mention of Stephens.

Now look at the second component of Brother Roper’s argument. “Someone” was interested in it, and because Joseph’s journal doesn’t indicate it was Joseph, Brother Roper concludes it must be Joseph. He relies on Joseph’s role as editor, for which the evidence is as threadbare as Brother Roper’s authorship claim. It’s fascinating that of all the things actually recorded in Joseph’s journal, only the things Brother Roper claims Joseph did are missing.

Brother Roper remains oblivious to how the evidence he cites supports my argument, not his. The fact that no one in Nauvoo mentioned these lengthy books, despite their (according to Brother Roper) tremendous importance, is evidence that it was not anyone living in Nauvoo who wrote the articles. It is not evidence that people simply forgot to mention Stephens (or Priest’s American Antiquities) in their journals.]

Figure 1: Discriminant Analysis Plot from “Joseph Smith, the Times and Seasons, and Central American Ruins.”¹¹ The Joseph Smith holographic texts, editorials signed “Joseph Smith,” editorials signed “Ed.” during his editorship, and the unsigned editorials during his editorship cluster together as a group and are obviously separate from the John Taylor and Wilford Woodruff texts. The editorials on the topic of Central America cluster closest to the Joseph Smith Group.

[Page 18]Joseph Smith’s journal does not record everything that Joseph Smith did or did not do during this time, but it does show that Joseph Smith was in or near Nauvoo while printing activities were going on, which is why he must be considered seriously as a candidate author.

[Another straw man argument. No one suggests the journals record everything Joseph did, but they do record what he deemed most important. He was in or near Nauvoo when horses were being shoed and meals were being prepared; does that mean he spent his time shoeing horses and cooking meals? Besides, it’s not merely the absence of Joseph’s alleged editing activities from his journal that is relevant. There is also third-party evidence to support what the journals indicate about Joseph’s activities; i.e., that he went to the printing office less than once a month while he was editor.]

Joseph Smith’s journal does show that he met with John Taylor on September 21 about the work of the printing office and also on September 23. This would have given him and John Taylor the opportunity to read or discuss what was to appear in the October 1 editorial and at least allow Joseph Smith to provide his own input, had he wished to do so.

[This is a misleading argument that underscores the utility of Joseph’s journal. What did Joseph and Taylor meet about? His journal tells us, and it has nothing to do with the content of the Times and Seasons! Joseph wanted to start another paper across the river. He told Taylor to have Woodruff go over and do it. I point out all of this in my book. A key point here is, what happened to those plans, and why? Read the book and find out, because Brother Roper won’t tell you. Instead, he helps make my point that Joseph could have provided his own input, had he wished to—but he didn't. Neither he nor John Taylor were talking about editorials or content of the Times and Seasons.]

Methods Used and Results: Our previous article examined the probable authorship of the Times and Seasons editorials related to Central America. A timeline of significant events, as well as other [Page 19]historical evidence, indicate that the most likely candidates are limited to Joseph Smith, John Taylor and Wilford Woodruff.

[This timeline consists of 7 points, and every one of them is false or misleading, as I’ve shown. I assume this is obvious to anyone familiar with Church history, but if not, maybe I’ll have to do a post on that, too.]

To characterize the writing style of each potential author, we assembled a collection of texts known to have been written by each author.

For Joseph Smith we took a sample of texts for which he is the known author (holographic documents written in his own hand) and combined them with texts from the Times and Seasons signed “Joseph Smith,” editorials signed “Ed” when Joseph Smith was editor, and unsigned editorials when he was the editor. We refer to these texts together as the “Joseph Smith Group.”

[This is problematic in multiple grounds. First, Brother Roper doesn’t list the documents, so we have no idea what he actually used. The one he does list—the Bernhisel letter—has an unknown author. Second, holographic writings are different in kind, purpose, and audience than published writings. Third, we know Joseph didn’t even write some of the material he signed—Woodruff noted that Joseph hardly had time to sign his name—so how can we legitimately include as his writings things we don’t know he wrote? Fourth, Joseph’s holographic writings date to the early 1830s. Such time disparity is a factor in assessing writing samples, as we will see. These problems are compounded by Brother Roper’s refusal to share with his readers what actual documents he included.

It turns out that Joseph’s holographic writings are collected by the Joseph Smith Papers here. If you go to that link you will see that there are only six items dated 1841-1843. Here’s the list:

Agreement with Ebenezer and Elender Wiggins, 14 May 1841

Note of Authorization, 24 February 1842

Complaint against William Thomas, 2 August 1842

Letter to Newel K. and Elizabeth Smith Whitney, 18 August 1842

Letter to Lucien Adams, 2 October 1843

Note to William Clayton, 9 December 1842

None of these—actually, none of Joseph’s holographic writings—include any mention of Central America, Stephens, or the like.]

We applied the statistical technique of discriminant analysis to identify how the texts group together based on the word-use frequencies in each text and then determined the probable group membership of the texts of unknown authorship.¹² 

[If you think this citation discloses any of Brother Roper’s data, such as what words were used to determine probably group membership, you’ll be disappointed. This is a citation to a useless article about theory.]

We showed (see Figure 1) that the Joseph Smith Group of texts cluster together, and they are distinct from the John Taylor texts and the Wilford Woodruff texts.

[I missed the description of the John Taylor and Wilford Woodruff texts. Did these also include holographic material? Articles signed by either of them but known to have been written by someone else? Articles not signed by either of them?]

The Central America editorials composited together

[In the original article, Brother Roper listed six editorials—three that included extracts from American Antiquities and three that included extracts from Stephens. But then he writes, “Our interest is the authorship of the five Times and Seasons editorials… To investigate probably authorship of the five small editorials (two signed “Ed.” and three unsigned) in the Times and Seasons that referred to Central America, we put them into one composite block of text so there would be sufficient data to measure word frequencies.”

Think about this for a moment.

First, Brother Roper arbitrarily excludes one of the editorials he identified because it doesn’t “refer” to Central America. (In fact, three of the editorials focus more on North America—Kentucky, Wisconsin and Tennessee—than Central America.)

Second, Brother Roper simply assumes that all five (or six) of the editorials were written by the same person. This is particularly interesting because three of them included extracts from the book that only Winchester owned.

Third, Brother Roper created this composite “so there would be sufficient data to measure word frequencies.” IOW, each individual article, by itself, is too short to measure word frequencies. Brother Roper doesn’t cite a theory of stylometry that justifies combining separate samples by unknown authors to determine the authorship of the composite.

Fourth, Brother Roper doesn’t tell us which portions of the editorials he used in the composite. This is important because quotation marks are optional in much of the Times and Seasons; other authors have confused some of the extracts with the editorial portions of the pieces.

It’s not clear from the narrative that Brother Roper even acknowledges these problems; certainly he doesn’t address them.]

are closest to the Joseph Smith Group of texts and obviously closer to those texts than to the John Taylor or Wilford Woodruff texts.

[This is meaningless “black-box” scholarship. So far, the stylometry analysis appears set up to confirm Joseph’s authorship, not to determine who wrote the anonymous editorials. Apparently we are expected to take Brother Roper’s word for all of this, but if he’s confident in the methodology and results, why the secrecy?]

Thus we concluded that the writing style in the Central American editorials is closest to the writing style of Joseph Smith, and consequently that Joseph Smith is the most likely author of the Central America editorials of the three historically justifiable candidate authors.

[These three, even if they did contribute to the Times and Seasons in 1842 beyond their signed contributions, were minor contributors. Apart from Joseph Smith’s History, the Book of Abraham, and D&C 127 & 128, Joseph wrote very little for the 1842 (3d edition) paper. Woodruff wrote one signed article, and John Taylor none. The main contributors to the third edition of the Times and Seasons were Orson Hyde (letters from his mission to Israel) and Benjamin Winchester (unsigned articles from the Gospel Reflector).  Most material published in the paper came through the mail, including the pseudonymous articles from Boston that I attribute, in part, to Winchester.

Even taking Brother Roper’s results on their face, the most that can be said is a mixture of unsigned editorials is closer to Joseph’s holographic writings than it is to Taylor’s and Woodruff’s published writings. But in what way? We have no idea and Brother Roper isn’t going to tell us.]

The “Lost City” of Zarahemla

In The Lost City of Zarahemla, Neville claims that the Central America editorials do not belong to the Joseph Smith Group and spends about 200 pages speculating how Winchester could have been the author. He repeatedly states his speculations as “fact” without scientific substantiation, and he even resorts to using a weak stylometric analysis for support.

[To the contrary, I offer a variety of other possible explanations for the historical data. I simply conclude that Winchester as author is the most plausible. I specifically questioned the utility of stylometric analysis generally, but I did apply the methods described by Lund (who at least tells readers what he does).

Later in this paper, Brother Roper will accuse me of using too many unspecific words, such as might and plausible. I could cite that part of his paper to refute his assertions above, but I trust readers can figure that out pretty easily.]

Neville’s Foundational Premise is Invalid

Neville asserts that there must be an unrecognized anonymous author for the Central America editorials by claiming that Figure 1 shows the Central America editorials collectively to be an “outlier” (pp. 219-20) in relation to the other texts in the Joseph Smith Group.

[Actually, I pointed to all three of the Figures Brother Roper provided. In this paper, he showed only the front view. In his original paper, he also showed the top and side views, both of which significantly separate the “Smith” group from the “Editor” group. Readers can decide whether Brother Roper was being informative by reproducing only the front view here. As noted previously, it’s always a good idea to check on the facts Brother Roper represents. You can see the other two Figures on page 96 here and decide for yourself.]

He says this leads him to believe that there was a different author other than Joseph Smith, [Page 20]John Taylor or Wilford Woodruff for the unsigned Central America editorials. Neville conjectures that Benjamin Winchester was that unrecognized latent author of the unsigned editorials.

What Neville means by “outlier” is more properly referred to as an “extreme value.”

[No, I did not mean an “extreme value.” I meant an “outlier.” Brother Roper’s database is a secret, but the brief description he has provided suggests fundamental problems, as I already listed. The “Central America” composite is closest to the “Editor” group (although it is still at some distance in 2 of the 3 Figures). In the Front and Top views, it is by far the furthest to the left of all the data points. Here’s how Brother Roper’s original paper characterized the relevance of distance in Figure 7:

A top view of the three authors’ texts showing the relative position of the Joseph Smith texts and the editorials with the texts signed “Editor” spread away from the other texts, indicating that those texts are somewhat distinguishable from the others. The Central America composite text is closest to the Joseph Smith text and the “Editor” texts.

In Figure 7, the “Central America” composite is farther from the “Smith” and “Editor” texts than the “Editor” text is from “Woodruff” or “Smith.” Brother Roper claims that distance renders those texts “somewhat distinguishable” from the others, but next he’s going to complain because I made the same observation about the Central America composite.] 

The term “outlier” refers to an extreme value among a set of values that is so far from the other data points that it is probably incongruent with the other members of that set of values. His method of assessment is purely visual; and, as he correctly says, it is only his “layman’s opinion” (p. 219).

[In his comment on his own Figure 7, Brother Roper described the relative position in visual terms. Apparently that’s okay because his visual method confirmed his bias. My application of his own approach is invalid because it contradicts his preferred belief.]

Yet his claim that the Central America editorials are an “outlier” is the foundational premise of his entire argument. If this claim is false, his entire argument has no basis and cannot be substantiated.

[This is another logical fallacy. Brother Roper is taking a common-sense visual observation that he can’t deny and converting it to a statistical term of art. And don’t forget, he’s also operating on the unsubstantiated assumption that all of the samples he characterized as written by Joseph were in fact written by Joseph and relevant to the question.]

Ignoring rigorous statistical analysis

[I was not ignoring Brother Roper’s own depiction of his data—I was relying on it. Anyone who reads Brother Roper’s article sees the same thing I do. He doesn’t provide any data other than his Figures, and refuses to provide me his underlying data, software, parameters, etc.]

 and only visually examining the plot in Figure 1, Neville concludes that the Central American texts are “too far” from the Joseph Smith Group to be congruent in style with the other texts. He does not realize that the distances shown in the plot are scaled relative to only the texts examined and do not represent any absolute measure of separation between the texts, as would be the case if the data points were simple two-dimensional locations of physical items plotted on a map in a Cartesian coordinate system.

[Brother Roper described the relative position in Figure 7 as sufficient to indicate that those texts “are somewhat distinguishable from the others.” I’m saying the even greater distance makes the “Central America” texts “somewhat distinguishable from the others.”]

Graphical Tests: His “eyeball method” is a simplistic approximation of applying a Euclidean distance measure like the distance-between-two-points calculation taught in high school algebra.¹³ Figure 2 shows a histogram plot (frequency plot) of the Euclidean distances of each text in the Joseph Smith Group from the centroid of the group.¹⁴ Along with these we have included the distances to typical points from the John Taylor and Wilford Woodruff groups to show what true “outliers” would look like on the plot. The points for John Taylor and Wilford Woodruff are extreme values, but not the Central America point. In fact, there is another text in the Joseph Smith Group even farther away than the Central America editorials.

[This analysis seems intent on undermining the validity of the original study—or at least Brother Roper’s explanation in that study. Of course, this should surprise no one; Brother Roper still hasn’t told us what he’s using as samples of Joseph’s writing, and there is reason to believe that much of what has been attributed to Joseph was actually written by others. That some of the other samples are even further away than the Central America editorials does not mean Joseph wrote them; it raises questions about the authorship of the comparison samples.]

Page 21]

Figure 2: Histogram of Euclidean Distances of Each Text in the Joseph Smith Group from the Centroid of the Group. The distances from the Joseph Smith Group centroid to typical points from the John Taylor group and the Wilford Woodruff group have been added to show what truly incongruent values look like. The Central America editorials are not inconsistent as part of the Joseph Smith Group, and there is even another Joseph Smith Group text more distant than the Central America editorials.

A data point exactly at the centroid of the Joseph Smith Group would have a Euclidean distance of zero (0.00). The first bar on the left in the plot shows five points with Euclidean distances between 0.00 and 0.50. The distances are deviations from the centroid and thus positive with no direction indicated. The next bar shows twelve points with distances between 0.50 and 1.00. An extreme value is one that is inconsistent with the rest of the data points in a set of data. For a point to be an extreme value in the plot, it would be to the right of the other Joseph Smith Group points, as indicated by a gap between it and the other group points. There is no such gap for the Central America editorials, which have a distance of 3.83, but there is an obvious gap in the distance to the typical John Taylor and Wilford Woodruff points, 6.08 and 6.10 respectively, showing what true extreme values look like. The histogram plot shows that the Central America editorials are within the distribution of the other points in the Joseph Smith Group and do not exhibit characteristics of an extreme value.¹⁵

[Only Brother Roper suggests that Woodruff and/or Taylor are the authors, so this is a red herring argument. So far, Brother Roper has declined to tell us not only what writing samples but the rest of his methodology. No one disputes the basic statistical analysis here, but it is impossible to dispute or agree with the relevance of what Roper has written here.

Here’s a short discussion of the problem with confirmation bias and data manipulation in the context of measuring and adjusting temperatures, but it applies here pretty well. Later, I’ll make the connection more explicit. Basically, it’s not the statistics that matter; it’s the underlying data that matter.

“[T]here is overwhelming evidence that confirmation bias doesn’t require anything like deliberate dishonesty. All it requires is a failure in applying double blind, placebo controlled reasoning in measurements. Ask any physician or medical researcher. It is almost impossible for the human mind not to select data in ways that confirm our biases if we don’t actively defeat it. It is as difficult as it is for humans to write down a random number sequence that is at all like an actual random number sequence (go on, try it, you’ll fail). There are a thousand small ways to make it so. Simply considering ten adjustments, trying out all of them on small subsets of the data, and consistently rejecting corrections that produce a change “with the wrong sign” compared to what you expect is enough. You can justify all six of the corrections you kept, but you couldn’t really justify not keeping the ones you reject. That will do it. In fact, if you truly believe that past temperatures are cooler than present ones, you will only look for hypotheses to test that lead to past cooling and won’t even try to think of those that might produce past warming (relative to the present).

“Why was NCDC even looking at ocean intake temperatures? Because the global temperature wasn’t doing what it was supposed to do! Why did Cowtan and Way look at arctic anomalies? Because temperatures there weren’t doing what they were supposed to be doing!

“One of the whole points about error analysis is that one expects a priori error from all sources to be random, not biased. One source of error might not be random, but another source of error might not be random as well, in the opposite direction. All it takes to introduce bias is to correct for all of the errors that are systematic in one direction, and not even notice sources of error that might work the other way. It is why correcting data before applying statistics to it, especially data correction by people who expect the data to point to some conclusion, is a place that angels rightfully fear to tread. Humans are greedy pattern matching engines, and it only takes one discovery of a four leaf clover correlated with winning the lottery to overwhelm all of the billions of four leaf clovers that exist but somehow don’t affect lottery odds in the minds of many individuals. We see fluffy sheep in the clouds, and Jesus on a burned piece of toast.

“But they aren’t really there.”]

[Page 22]A “box plot” is another standard statistical representation of a set of data.¹⁶ The box plot shown in Figure 3 further illustrates that the Central America editorials are not an extreme value of the Joseph Smith Group.

The “box” contains the middle 50% of the data points. The vertical line extending down from the box spans the lower 25% of the data, ending at the minimum value. The vertical line extending upwards from the box spans the upper 25% of the data, ending at the maximum value.

The threshold distance for an extreme value for the Joseph Smith Group using typical box plot methodology is 4.41. The Central America editorials distance of 3.83 is not beyond the threshold. Therefore, here again, the Central America editorials are not an extreme value within the Joseph Smith Group of texts.

[Brother Roper has substituted his own argument for mine. He’s arguing with himself. I never said the articles represented an extreme statistical value. Brother Roper himself, in his article, noted that the “Central America” text fell outside the grouping of his Joseph material, which he interpreted to mean John Taylor helped with the writing or editing. Now he is arguing with his own initial article, not my point.]

Figure 3: Box Plot of Euclidean Distances from the Centroid of Joseph Smith Group with the Extreme Value Threshold Distance. The Central America editorials distance is not an extreme value in relation to the other texts in the Joseph Smith Group.

Univariate Tests: There are numerous objective statistical tests for extreme values in a set of data: Dixon’s Q Test, Grubb’s Test, Iglewicz and Hoaglin Test, Rosner’s Generalized Extreme Studentized Deviate [Page 23](GESD) Test, and Tietjen-Moore Test.¹⁷ To further test Neville’s eyeball method, we applied each of these statistical tests to the two-dimensional distance data in Figure 1. Table 1 shows the results.

[All of this ignores the point that we still don’t know 1) Brother Roper’s writing samples or 2) his methodology. He’s running statistical analysis without first defending—or even explaining—the underlying data. This entire exercise is pointless. If anything, it illustrates the fundamental weakness of Brother Roper’s argument: he has no historical evidence to support his assertion, so he’s misleading readers by using statistical analysis purely on the assumption that his underlying data is valid.]

Extreme Value Test	Central America’s Test Value	Extreme Value Criteria	Result
Dixon’s Q	0.115	>0.206	Not Extreme
Grubb’s	2.379	>3.230	Not Extreme
Iglewicz Y Hoaglin	2.343	>3.500	Not Extreme
Rosner’s GESD	2.168	>2.768	Not Extreme
Tietjen-Moore	0.841	<0.751	Not Extreme

Table 1: Results of Extreme Value Tests of the Euclidean Distances in Figure 1. All the tests show no evidence that the Central America editorials are an extreme value within the Joseph Smith Group.

Each test uses the data and calculates a test value for the Central America editorials. The test value is compared to a criterion value that indicates whether the Central America editorials might be an extreme value or not. For example, Dixon’s Q test calculates a test value for the Central America editorials of 0.116. This is compared to the criterion value of 0.206. Since 0.116 is not greater than 0.206, there is no evidence that the Central America editorials are an extreme value. For the first four tests in Table 1, if the calculated test value is greater than the criterion value, then that would indicate an extreme value. The Tietjen-Moore test is different. If its test value is less than the criterion value then this would indicate an extreme value.

As we can easily see in Table 1, all the tests show no evidence that the Central America editorials are an extreme value from the other texts in the Joseph Smith Group.

Multivariate Test: Since the data measure the proportions each author used 67 noncontextual words, the data constitute a 67-dimensional multivariate data set. To visualize the multi-dimensional data in graphical form on paper we needed to depict it in our previous article in only two dimensions. The amateur Neville sees two-dimensional plots and thinks this is all the information. Consequently, Neville was easily [Page 24]deceived by his “eyeball test.”

[This is what I observed at the outset was funny. I actually requested the underlying data; Brother Roper refused to share it with me. Consequently, I was left with only what was reported in the original paper. Now they criticize me for not having analyzed the data they refused to provide me! In that article, Brother Roper gives his readers two-dimensional depictions of three-dimensional data, with none of the underlying data. If I was “deceived” it was by Brother Roper’s own interpretation, in that article, of his own data. Worse, so far in this article, he has provided us with only one two-dimensional depiction.]

The most commonly used test for extreme values within a high-dimensional multivariate data set is to test the Mahalanobis distance.¹⁸ For a Mahalanobis distance to be considered an extreme value, the random sampling chance of observing that distance is generally required to be less than one in a thousand (probability < 0.001).¹⁹ Using the data from our previous article, this test shows that the Central America editorials are not extreme values in comparison to all the other texts in the Joseph Smith Group. The results are shown in Table 2.

	Joseph Smith	John Taylor	Wilford Woodruff	Critical Value
Mahalnobis Distance	12.79	72.80	100.74	13.82
Group Membership Probability	100.0%	0.0%	0.0%	>95%

Table 2. Mahalanobis Distances and Probability of Group Membership for the Central America Editorials. The Mahalanobis distance from the editorials to the centroid of the Joseph Smith Group is not beyond the critical value. The probability of the editorials’ membership in the Joseph Smith Group is virtually 100%.

Since the Mahalanobis distance from the Central America editorials to the centroid of the Joseph Smith Group (12.79) is not larger than the critical value (13.82), while the distances to the centroids of the John Taylor and Wilford Woodruff groups are larger than the critical value, the editorials are judged to be “outliers” from John Taylor and Wilford Woodruff but not from Joseph Smith. The probability of group membership of the editorials in the Joseph Smith Group is virtually 100%. This further indicates that there is no evidence the Central America editorials are “outliers” in the Joseph Smith Group when tested with the appropriate statistical technique.

[Here again, Brother Roper is assessing his own argument, not mine.]

Open-Set Test: Is There Evidence of a Latent Candidate Author? A statistical technique that can be applied directly to Neville’s claim of someone other than Joseph Smith, John Taylor or Wilford Woodruff[Page 25]authoring the three unsigned Central America editorials is the Extended Nearest Shrunken Centroid Method (ENSCM). ENSCM is an extension of a sophisticated technique developed for high-dimensional classification problems in genomics research and DNA microarray analysis that we and others have used in authorship attribution, including our studies of the Book of Mormon.^{20 21 22 23 24}

[These studies are hardly uncontroversial and unchallenged, either. Mostly they’re ignored. No offense, but I think this article is another reason why Brother Roper’s analyses should be ignored.]

Using ENSCM, we tested Joseph Smith, John Taylor and Wilford Woodruff as an open-set of candidate authors of the Central American editorials. ENSCM first establishes a profile of word-use frequencies for each candidate author based on texts he or she is known to have written.

[Maybe I missed the detailed list of the texts tested here and the profile of word-use frequencies. Surely Brother Roper wouldn’t still be keeping that from us, would he?]

Then ENSCM computes the probability that each candidate author’s writing style is closest in style to the style of the texts of unknown authorship. However, ENSCM allows for the possibility of an additional unknown latent author — sometimes referred to as the “none of the above” alternative. Should the latent author’s probability of closest writing style exceed the probability of any of the candidate authors, then the group of authors should be considered to be an open-set and include the possibility of an unknown author.

Applying ENSCM, we found that the latent author probability — the probability someone else needs to be considered as having a writing style closer than at least one of the candidate authors — is less than one in a thousand (probability < 0.001). See Figure 4. This means that the word-use frequencies of the candidate authors are close enough to the word-use frequencies in the Central America editorials that we can conclude there is insufficient evidence of the need to consider other authors. Consistent with the historical evidence, Joseph Smith, John Taylor and [Page 26]Wilford Woodruff can be considered to be a closed-set of candidates for authorship of the three Central America editorials.

Consequently, in our previous paper we included only Joseph Smith (and the editorials that group with Joseph Smith), John Taylor, and Wilford Woodruff in the analysis. Since the question of authorship of the Central American editorials could be addressed as a closed-set, the evidence indicates that Joseph Smith is the most probable author, as we concluded in the previous paper.

Figure 4: Extended Nearest Shrunken Centroid Method (ENSCM) Probability of Closest Pattern. The probability of a latent author is less than 0.001. This indicates that Joseph Smith, John Taylor and Wilford Woodruff (along with “Ed.”) can be considered as a closed-set of candidates for attributing authorship of the Central America editorials.

Thus, Neville’s biased “layman’s opinion” based on his “eyeball test” is contradicted by a host of relevant objective statistical tests and analyses. Therefore, his assertion that another author needs to be considered is baseless. Consequently, the entire foundational premise of Neville’s book is invalid.

[Brother Roper’s statistical analysis was not the foundational premise of the book; the question of who wrote the articles was. Brother Roper’s article was so full of historical errors that I questioned its validity on that basis alone. On top of that, his portrayal of the 3-dimensional data appeared to contradict his own conclusion—and it still does. But to the merits of this ENSCM analysis, it’s a classic case of garbage in, garbage out. Until Brother Roper fully discloses his data, software, assumptions, parameters and methodology, he is reporting on nothing more than a Black Box outcome. That’s ideal for confirmation bias, but entirely unpersuasive for anything else.]

Adding Benjamin Winchester to the Analysis

We have shown that the foundational premise of Neville’s argument is invalid; nevertheless, to directly test Neville’s contention, we added Winchester to the [still undisclosed] data from our 2013 article and reanalyzed the data to see how close his style is to the style in the [arbitrarily composited] Central America editorials. We also tested for evidence that his style is closer to the editorials than to Joseph Smith’s. We followed objective, formal [and still secret] scientific hypothesis-testing methodology.

We formulate the research question as follows:

[Page 27]Is Benjamin Winchester’s writing style the same as the writing style in the Central America editorials, and is his style closer to that of the editorials than to Joseph Smith’s style?

To answer this research question we formulate the null (H₀) and alternative (H_a) hypotheses as follows:

H₀: The Central America editorials writing style is closest to Joseph Smith’s style. [using still secret writing samples]

H_a: The Central America editorials writing style is closest to Benjamin Winchester’s style. [using still secret writing samples]

We performed [still secret] discriminant analysis and determined the probabilities of [still secret] group membership. Figure 5 shows a plot of a discriminant analysis similar to that in Figure 1 with Winchester added as a candidate author.

Figure 5: Discriminant Analysis, including Winchester. The Central America editorials are closer to those of the Joseph Smith Group than to the Winchester texts.

The first discriminant function (the dimension of greatest distinctiveness) differentiates Winchester from the other three authors. The second function differentiates Joseph Smith from Wilford Woodruff.

[Setting aside the irony of Brother Roper resorting to this visual analysis after criticizing me for doing the same thing, Figure 5 is quite revealing.

First, Brother Roper is showing us he’s running a different analysis than he did in his original paper; none of the three Figures from the original paper showed the results that appear in Figure 5. His depiction of Discriminant Function 1 vs 2 in Figure 1 of this very paper is much different from what we have in Figure 5. This leads me to wonder what the analysis would look like if he simply plugged Winchester into the analysis he did in the original paper.

Second, there are far more “Joseph Smith” data points in Figure 5 than in Figure 1, suggesting that Brother Roper has converted some of the “Editor” text into “Joseph Smith” text. This is particularly significant because in Figure 1, it was the Editor texts that were closest to the “Central America” composite.

Third, there are far more “Joseph Smith” data points than “Benjamin Winchester” data points, and they are more spread out. Making the database for “Joseph Smith” larger than for the other candidates makes the “Joseph Smith” circle much larger as well. Had Brother Roper used the same number of data points for “Joseph Smith” as he did for the other candidates, “Joseph Smith” would be a smaller circle and may have been farther from “Central America” than “Winchester,” depending on which data points Brother Roper retained. IOW, Figure 5 shows how this type of analysis can be gamed to get the “right” result.

Fourth, even with this questionable and secret database, Winchester is closer to the “Central America” text than either Taylor or Woodruff in both Discriminant Functions. Of all four candidates, Winchester is the closest in Discriminant Function 1.

Fifth, Brother Roper offers only one “view” of the data in Figure 1 in this paper (Figure 6 in his original paper) because Figures 7 and 8 in his original paper showed much greater distance between the “Central America” text and the “Smith” texts. Now he’s showing only one view in Figure 5. Is it unreasonable to surmise this is because the other two views also show greater distance between “Central America” and Smith” like they did in his first paper?]

The Central America editorials clearly cluster with the Joseph Smith Group and not with Winchester.

[This is a questionable claim for the reasons noted above, but recall that in his original paper, Brother Roper commented on Figure 8 by writing, “John Taylor’s texts are about as close on average as Joseph Smith’s texts, indicating John Taylor’s possible influence in writing the texts signed ‘Editor.’” If we look at Discriminant Function 1 in Figure 5 above, the closest data point is Winchester’s. None of Winchester’s data points are as far away from Central America as about 1/3 of the Joseph Smith points. If Brother Roper is consistent, he would have to conclude that this indicates Winchester’s “possible influence in writing the texts” labeled “Central America.”]

Even Neville’s “eyeball test” would[Page 28]conclude that the Central America editorials are an “outlier” relative to the Benjamin Winchester texts rather than the Joseph Smith Group of texts. To make matters worse for Neville’s eyeball, the third function, which is not shown in the two-dimensional plot in Figure 5, separates John Taylor from the others and moves Winchester even further from the Central America editorials.

Applying the appropriate statistical distance measure for multivariate data — the Mahalanobis distance — the evidence shows the Central America editorials to be an “outlier” from the Benjamin Winchester, John Taylor and Wilford Woodruff groups, but not from the Joseph Smith Group. This is shown in Table 3.

	Joseph Smith	Benjamin Winchester	John Taylor	Wilford Woodruff	Critical Value
Mahalnobis Distance	15.97	25.48	45.61	54.35	16.27
Group Membership Probability	98.9%	1.1%	0.0%	0.0%	>95%

Table 3. Mahalanobis Distances and Probability of Group Membership for the Central America Editorials. The Mahalanobis distance from the editorials to the centroid of the Joseph Smith Group is not beyond the critical value. The probability of the editorials’ membership in the Joseph Smith Group is 98.9%.

Since the Mahalanobis distance from the Central America editorials to the centroid of the Joseph Smith Group (15.97) is not larger than the critical value (16.27), while the distances to the centroids of the Benjamin Winchester, John Taylor and Wilford Woodruff groups are larger than the critical value, the editorials can be judged to be “outliers” from the others, but not from Joseph Smith. Of the four, Joseph Smith is a much more likely candidate as author of the Central America editorials. The probability of group membership of the editorials with the Benjamin Winchester text is a mere 1.1%. This shows even more lack of evidence contrary to the null hypothesis.

[All of this is garbage in, garbage out, for the reasons listed above. No amount of statistical manipulation can offset biased (or, worse, secret) data.]

We could end the article here, since we have fully demonstrated that Winchester is not a more likely author of the Central America editorials than Joseph Smith. But, since Neville spends hundreds of pages trying to build his case, it is necessary to analyze his methods more deeply. We will discuss Neville’s “pseudo-stylometry,” and then to show further how poor a candidate Winchester is, we compare him to an expanded pool of candidate authors

[The article might as well end here—the rest of the article gets even worse. So far, Brother Roper has still not explained either his data or his methodology.][Page 29]

Stylometry: The Statistical Analyses of Writing Style

Our previous article and this article use stylometry: the statistical analyses of written text to characterize the writing style of the author. In authorship attribution it is necessary to first examine the historical evidence for authorship. Without a solid historical foundation, attribution assertions are baseless. Yet, if we consider only the historical evidence, we can get only so far towards an answer, since multiple scenarios could still remain plausible. Applying stylometrics, if done correctly, can provide additional information showing who the most likely author is, given the historical context.

Stylometry uses statistical measures to characterize an author’s writing style and identify what makes it unique from other authors’ styles. Many approaches have been used to define an author’s unique writing style. All have looked at various features of an author’s writing as measures of style. Some have counted letters, words, word-pair choices, unique words, word lengths, sentence length, paragraph length, language complexity, and many other metrics of style in an attempt to distinguish one author from another. Some have proven to be better than others.

A good metric of writing style is one that is consistent within an author’s writing and yet different from that of other authors. Many naïve methods are not capable of meeting these requirements. Among these deficient methods are average sentence length, unique words, and language complexity. However, some methods are capable of satisfying the criteria.

Focusing on what are called noncontextual words has been shown to be highly useful. Examples of noncontextual words are the words and, for, of,the, and to. These are function words — they do not convey the author’s message but provide the structure by which the author forms his or her message. They define the grammatical relationships among words instead of conveying specific information themselves.

Noncontextual function words are used by all authors, but not in the same way or with the same frequencies. Therefore, different usage frequencies for noncontextual words are useful in characterizing an author’s subconscious word “fingerprint,” sometimes referred to as his or her wordprint. Consequently, the use of noncontextual words is a standard approach in the field of stylometry. Although the specific noncontextual words that are distinguishing among authors vary from [Page 30]study to study, their effectiveness in measuring writing style is well established.²⁵

It is best to select the noncontextual words for a specific study that truly distinguish the authors in that study.

[This is circular reasoning, as I’ll show soon.]

If this is not done, then the words selected may not be the ones that will show differences among the writing styles in the study.

[IOW, if you don’t choose the noncontextual words that “truly distinguish” the authors, then the words you select won’t show the differences. This needs to be explained? Twice?]

Once the noncontextual words for a study have been selected, an appropriate analysis method must be used. Discriminant analysis is well-suited to the requirements for a good stylistic measure because it can find the combination of weights for the words that (1) best shows consistency of word-use within authors, and (2) at the same time discriminates among different authors’ word-use tendencies.

Discriminant analysis is the method we used in our previous article and one of the methods used in this article. Studies that do not use powerful validated statistical methods are deficient and prone to yield misleading, unsupportable conclusions. Neville’s methodological approach is a textbook example of how not to do stylometric research and the consequences of doing it inappropriately.

[I have to say, this is a ludicrous criticism. I don’t think much of stylometry, especially Black Box stylometry, and I thought I made that clear in the book. Brother Roper declined my request for his data and methodology; I didn’t even attempt to guess what he did, other than to look at the meager data he published in the form of his Figures. My “methodological approach” consisted primarily of showing how Lund’s approach tended to support Winchester as author. That wasn’t an endorsement of the approach, but an illustration of how, even with that approach, Lund disproved Joseph was the author.]

Neville’s Pseudo-Stylometry

Noted authorship attribution historian Harold Love says, “Anyone wishing to conduct serious research in attribution studies cannot do so today without a good general understanding of the nature and basic techniques of statistical reasoning.”²⁶ Neville, lacking such an understanding, presents the unwary reader with several pseudo-stylometric analyses which he claims provide evidence in support of his Winchester-authorship theory. The main ones he uses are average sentence length measured as average number of words per sentence, words unique to one author compared to those of other candidate authors, and “cherry-picked” word-pattern similarities. All these methods are amateurish, nondistinguishing techniques. We examine Neville’s use of these in detail and show their deficiencies.

[This is laughable. I explained that I am skeptical of stylometry generally, and I assessed both Roper and Lund on this point. Here, I applied Lund’s methodology. To the extent Roper agrees with me that Lund’s analysis is flawed, I’m fine with that. But I wasn’t using Lund’s analysis to prove Winchester was the author, only to prove that using Lund’s own methodology, Winchester was a better fit.]

Average Sentence Length: Neville compares Winchester’s average sentence length (ASL) to the ASL of the Central America editorials, [Page 31]showing they are about the same (p. 225). However, ASL is not a good measure of style by the two criteria for a good style metric: consistency within an author and differentiation among authors.

ASL was first used to attribute possible authorship over one hundred years ago.²⁷ It is an archaic method that has been shown to be unreliable and nondistinguishing. A specialist in disputed authorship of documents, Patrick Joula, says, “Many other statistics have been proposed and largely discarded, including average sentence length.”²⁸ Naïvely using ASL can lead to faulty conclusions and self-deception.

[So far, all of this is consistent with my critique of Lund and stylometry generally. Brother Roper here is making a classic red herring argument. This section of the paper represents an enormous waste of time by Brother Roper.]

To show the deficiency of ASL to distinguish between authors we use The Federalist Papers, commonly used for testing the usefulness of authorship-attribution methods.²⁹ Well-known stylometrician David I. Holmes says, “The Federalist problem has been used … as stylometry’s ‘testing ground’ for new techniques.”³⁰

The authorship of twelve of the eighty-five Federalist Papers has been disputed, but stylometric analyses have shown that they were all probably written by Madison, with the possible exception of one paper. However, attempting to identify the author of the disputed Federalist Papers using ASL proves problematic. Figure 6 shows the ranges (lowest to highest) of ASLs for the papers commonly attributed to Hamilton, Madison and Jay, along with the range of ASLs for the disputed papers (with Winchester added as a comparative control). In order to be comparable, Winchester’s [secret] texts were concatenated and split into blocks of text that were about the length of the average size of The Federalist Papers (2058 words).

Based on the ASLs, although Jay might be ruled out as the author of the disputed papers, it would appear that Hamilton may be a better choice than Madison, but the difference is small and unconvincing. Using ASLs as a method of author identification fails to identify the author of the disputed Federalist Papers.[Page 32]

Figure 6: Ranges of Average Sentence Length (ASL) for The Federalist Papers with Winchester Added for Comparison. The ASLs vary widely within authors and do not provide a basis to make convincing conclusions about authorship of the disputed Federalist Papers. Winchester appears to be the best choice as the author of the disputed papers — a clear sign that the method is inadequate.

Furthermore, ASL as a measure of writing style fails to distinguish the control author (Winchester) from the other candidates. In fact, Winchester’s ASL matches the range of ASLs of the disputed Federalist Papers more closely than any of the actual Federalist Papers authors, but Winchester had not yet been born when The Federalist Papers were written. So using ASL can lead to absurd conclusions for The Federalist Papers. It is equally not useful when Neville applies it to the Times and Seasons editorials. Based on ASL, Winchester is more likely to be the author of the disputed Federalist Papers than he is to be author of the Central America editorials.

[Not only is Brother Roper making a red herring argument (he’s attacking Lund, not me), but he’s displaying the fundamental problem with his own analysis. Without historical context, stylometry has to include every potential author. Brother Roper’s historical analysis is deeply flawed from every direction, starting with his assumption that the author had to be residing in Nauvoo.  There is an unlimited universe of potential authors if one ignores the historical context.]

Words “Unique” to an Author: Neville focuses on an author’s “unique” words, i.e., words which he claims one candidate author uses but which the other candidate authors do not use. Such words are sometimes referred to as “marker” words. However, stylometrician Leon Maurer notes, “It turns out that rare words do not provide as reliable a ‘fingerprint’ because, while it is easy to work in certain words now and then, it is hard to change personal modes of common word use.”³¹

[This objection raises the issue of time disparity, which I suspect I’ll return to shortly. But the “marker” words objection is a red herring, too, as I’ll demonstrate.]

Again using The Federalist Papers as a standard to evaluate Neville’s technique, we find that 16% of Madison’s unique words are also in the disputed papers, 14% of Hamilton’s unique words are in the disputed[Page 33]papers, and 5% of Jay’s unique words are in the disputed papers. Adding Winchester to the mix as a control, we find that he has 3% of his unique words (which is twenty “marker” words) in the disputed papers. Although 3% is less than the percentages of Madison, Hamilton and Jay, Neville did not use percent but only pointed out that there were some of Winchester’s unique words in the Central America editorials. By his approach, we ought to conclude — as Neville’s approach would require — that Winchester wrote the disputed Federalist Papers simply because the disputed papers contain some of his unique words!

[This would be a useful observation if the authors shared with us what “unique” words they used. Whatever secret words Brother Roper chose here, evidently they are not actually unique; Brother Roper finds them in the papers written by other authors. And, of course, the historical context is key, as Brother Roper previously stated: “Without a solid historical foundation, attribution assertions are baseless.”]

Repeating this exercise using Neville’s words in The Lost City of Zarahemla, we find that Neville has forty-two unique words that appear in the disputed Federalist Papers. Neville’s method of pointing out unique words used in the Central America editorials, and saying that that provides evidence of authorship, would require him to conclude that he himself had written the disputed Federalist Papers. The use of unique so-called “marker” words fails to distinguish clearly the author of the disputed papers, and Neville’s way of using them would not eliminate the control, Winchester, nor would it even eliminate himself as the author.

[This is the kind of argument that passed for “scholarly” in FARMS and Maxwell Institute papers, so it’s no surprise to find it in the Interpreter. Note that I at least explained my methodology; Brother Roper doesn’t.

At any rate, I understand why Brother Roper avoids the use of marker words analysis, and I’ll demonstrate that below.

But two of the authors Brother Roper cites in this paper have used “marker” words on the Federalist Papers with success. “Mosteller and Wallace identified a list of 30 “marker” words, found to be more typical of either Hamilton or Madison’s undisputed writings. They combined the evidence of how often each of these words occurred in each disputed paper using Bayes’ rule, and found that the odds on each paper having been written by Madison were at least 240 to 1.” So contrary to what Brother Roper wants you to believe, using marker words can be effective.]

So using Neville’s method leads to useless results. Thus applying Neville’s “unique words” method is not reliable and cannot provide useful support for his Winchester conjecture.

[Roper’s own citation says, “rare words do not provide as reliable a ‘fingerprint’ because, while it is easy to work in certain words now and then, it is hard to change personal modes of common word use.” This is not the same as “useless,” and the utility depends on the words and the proximity in time of the sampled writings.]

Cherry-picked Word Pattern Similarities: Neville singles out 73 words and phrases in the Central America editorials and asserts that they are similar to words and phrases used by Winchester (pp. 207‒16).

[So far, this is 73 more words of detail than Brother Roper has provided. Besides, he is assessing the First Edition of the book, which he knows was preliminary—and intended to invite further research. The Second Edition is far more detailed on this point.] However, his statements rely only on mere circumstantial similarities. Many people use words similar to the words in the Central America editorials simply because the words are commonly used by English speakers.

[Well, a word such as “Zarahemla” is hardly “commonly used,” but it was used 4 times in the Oct. 1 article. The word was used only 9 times in the entire Volume 3 of the Times and Seasons (24 issues). Once it was used in a reference to the title of Don Carlos proposed newspaper; once in reference to a marriage that took place in Iowa; three times in an article by Benjamin Winchester; and four times in the Oct. 1 Zarahemla article.

In fact, there are only two articles in the history of the Times and Seasons that refer to Zarahemla more than once (not counting an Iowa conference report): Winchester’s article, and the Oct. 1 article.

It’s understandable that Brother Roper would dismiss such usage as “common to English speakers,” and typically, he provides no evidence for his assertion, but I’ll ask readers to think about just how common it was for English speakers in 1842 to refer to Zarahemla multiple times in a single short article. If Brother Roper told us what writing samples he used, we could see how many contained this term. One word that never shows up in Joseph Smith’s holographic writing, for example, is Zarahamla.]

At best the word pattern similarities only show that Winchester and the author of the three editorials were speaking English.

[At best? This article is replete with this type of irrational, counterfactual statements. I haven’t bothered highlighting all of them, but this one in particular deserves a mention.

First, the word patterns show the author was writing English, not speaking it.

Second, unlike the case with Joseph Smith, we have actual articles written by Winchester on this specific topic. So the “word pattern similarities” Brother Roper dismisses as irrelevant show a mutual interest in this topic between Winchester and the anonymous author of the 900 words. The similarities are also contemporary; Winchester wrote in 1841 and 1842, while the 900 words were published in 1842.

The same cannot be said about Joseph Smith, despite Brother Roper’s wishful thinking. There is not a single statement that can be unambiguously tied to Joseph Smith that includes the words “Stephens,” “Central America,” Guatemala,” or any of the other topic-specific terms from the 900 words. If Joseph wrote these 900 words, then they are unique in all of his lifetime work.]

Contextually similar phrases are insufficient to attribute authorship. Such similarities can only generate a question about authorship, not an answer.

[This is a nonsensical statement. No one claims that Person A wrote a particular article just because he has written about the same topic in the past, but if he/she has never written about the topic, and has never used the terms associated with that topic, that person can be excluded as a plausible author. Take the Federalist Papers as an example. Did the stylometrists evaluating those papers check the writings of chemists or blacksmiths of the day who had never written a word about politics? Of course not. Topical relevance is sufficient to exclude entire categories of potential authors and, if the topic is narrow enough, can point to a limited group of authors who have written on the topic.]

As we have noted, noncontextual words rather than contextual words are well-recognized among stylometricians as useful in characterizing authorial styles, but all the words and phrases Neville focuses on are contextual words.

[Noncontextual and contextual words are both useful tools. Brother Roper excludes contextual words because they eliminate Joseph from contention, not because they are not useful and probative.]

Contextual words are not as much an indication of an author’s style as an indication of the author’s subject matter. None of the 73 words and phrases that Neville focuses on is distinctive of Winchester’s writing style as opposed to other authors’ styles.

[The list of authors in the 1842 time frame who wrote on this topic is finite. First, they had to be LDS; no one else was trying to prove the divine authenticity of the Book of Mormon. Second, they had to be writing about Central America. That limits the universe to Benjamin Winchester, Orson and Parley Pratt, John Page, William Smith, and W.W. Phelps. If there are others, add them to the list. (Brother Roper likes to add Woodruff and Taylor, but neither of them published anything on the topic. At most, Brother Roper can claim they wrote anonymous articles, but that begs the question.) Third, they had to be published in the Times and Seasons. That excludes Page. It makes it very unlikely for the Pratt brothers in 1842 for practical reasons. Plus, neither of them was known for writing anonymously. That leaves only Winchester, William Smith, and Phelps as plausible candidates based on style and contextual words.]

He tries to make Winchester’s use of the 73 words mean something, when they are not distinguishing among other English-speaking authors to begin with. [Page 34]His method is saying, in effect, “See, the author of the Central America editorials uses these words and so does Winchester.” When in reality so do many other English-speaking people.

[Brother Roper keeps repeating this point but the repetition doesn’t help. If anything, he highlights the contrast between my analysis, which is completely open and specific, and his analysis which is completely closed and vague. I invite replication and collaboration; Brother Roper shuns it. On the merits, though, what “other English-speaking authors” were writing about Stephens and Zarahemla? Especially around 1842? If they exist, Brother Roper needs to list them so we can discuss them.]

If we use Neville’s similarities between Winchester and the Central America editorials and apply discriminate analysis with Joseph Smith and John Taylor included, the evidence still indicates that Joseph Smith is more likely to be the author of the editorials due to stronger similarities than Winchester.

[This is sleight-of-hand. Brother Roper has expressly rejected a contextual analysis. The discriminate analysis he purportedly conducted was based on a secret database with all the problems I already mentioned.]

So we must conclude that there is no evidence that Winchester is the author of the editorials based on Neville’s cherry-picked similarities.

[In the Second Edition, I go through each term and phrase in the 900 words. No cherry picking at all. I look at everything. That’s how I ended up with Phelps’ participation.]

Conclusion about Neville’s Attempt at Stylometry: In sum, the pseudo-stylometric analysis done by Neville is unreliable. He relies on average sentence length, so-called “unique” marker words, and cherry-picked similarities, all of which have been shown to be nondistinguishing.

[Anyone reading my first edition knows I’m skeptical of stylometry and particularly “black box” work such as Brother Roper’s and crude methods such as Brother Lund’s. Brother Roper’s own citation above contradicts his assertion that unique words are non-distinguishing, and I cited one article by authorities he cited to show that “marker” words are useful to determining authorship.]

Neville’s “layman’s” observations, analytic methods and reasoning result in unfounded, misleading, erroneous conclusions. They do not provide valid support for his Winchester authorship theory. For further consideration, Neville’s pseudo-stylometric analysis is evaluated in even greater detail in the Appendix.

[Brother Roper keeps repeating his argument, but the repetition only accentuates the fallacies of his own approach. I’ve shown how his historical assumptions are unfounded and misleading. I’ve shown how his stylometry analysis uses secret, questionable data with secret parameters and assumptions, leading to a Black Box result. No matter how sophisticated the statistical analysis, if the data is biased or irrelevant, the analysis is a failure. In the next section, Brother Roper is going to demonstrate that his methods are pure confirmation bias.]

Appropriate Stylometric Analysis

If Benjamin Winchester should be considered as a candidate, perhaps there are other early LDS writers who also should be considered. Up to this point all the analyses we have shown use the data from our previous article. We transition now to performing stylometry, using an expanded set of comparison authors.

To do stylometry appropriately one needs an appropriate set of authors, focused texts, truly distinguishing features to analyze, and high-powered methods to rule out unlikely candidates. Objective formal scientific hypothesis testing methodology should be used.

How We Picked an Expanded Comparison Set of Authors: Although we contend, on the basis of historical and statistical evidence, that the Central America editorials authorship question is a closed-set problem, to directly test Neville’s assertions we stylometrically evaluated Winchester as a candidate author among an expanded comparison group of authors with historical backgrounds that make them potentially plausible authors and who published writings about American antiquities comparable in subject matter to the unsigned Times and Seasons editorials during the same period of time in Church history.[Page 35]

[Good. I welcome this.]

Expanded List of Other Potential Candidate Authors for the Unsigned Editorials: In addition to Joseph Smith, John Taylor, and Wilford Woodruff, it might be conjectured based on historical evidence alone, that the unsigned editorials were written by other members of the Church who are known to have written about the Book of Mormon and American antiquities previous to the Times and Seasons editorials. These include George J. Adams, John E. Page, W. W. Phelps, Orson Pratt, Parley P. Pratt, William Smith, Erastus Snow, Charles B. Thompson, and Benjamin Winchester.

[Okay, but I’d like to know where Adams, Page, Snow and Thompson were published in the Times and Seasons. Snow was published along with Winchester. Articles about Adams were published (including two written by Winchester). A review of Thompson’s book was published (probably written by Winchester). But I’m happy to include all of them.]

Although ENSCM showed that Joseph Smith, John Taylor, and Wilford Woodruff could be considered a closed set, we compared Winchester to these others to see if he is the closest in style among them to the style in the Central America editorials. If he is not, then his viability as a candidate for authorship diminishes even more.

George J Adams. George J. Adams was an actor who joined the Church in New York in early 1840. His flamboyance and skills as an orator were often used in defense of the Church in the Eastern States, and England, and reports [newspaper reports mailed to Nauvoo, including four written under pseudonyms of Winchester and John Eaton] of his debates with opponents were often printed in the Times and Seasons and the Millennial Star. He published several pamphlets in defense of the Mormons.³² Adams visited Nauvoo in September 1842. On September 7 the Prophet’s journal records:

Early this morning Elder Adams and brother Rogers from New York visited president Joseph and brought several letters from some of the brethren in that region. … In the P.M. brother Adams & Rogers came to visit him again. They conversed upon the present persecution &c president Joseph in the discourse to brothers Adams and Rogers shewed the many great interpositions of the Almighty in his behalf not only during the present trouble, but more especially during the persecution in Missouri &c. The remarks droped on this occasion was truly encouraging and calculated to increase the confidence of those present.³³

[Page 36]The two men again visited the Prophet five days later on September 12: “At home all day in company with brothers [George J.] Adams & [David] Rogers, and councilling brother Adams to write a letter to the Governor.”³⁴The Prophet sat to have his portrait painted by Rogers at his home on September 16, 17, 19 and 20.³⁵ Following his excommunication in 1845, Adams followed the leadership of James Strang, organized his own church in 1861, and led an ill-fated attempt to settle in the Holy Land in 1866.³⁶

[More evidence that they weren’t discussing Stephens and Central America. Besides, I’m still waiting for evidence that Adams ever published anything in the Times and Seasons.]

John E. Page. John E. Page, baptized in 1833 and ordained an apostle in 1838, received a call to accompany Orson Hyde on a mission to Holy Land, but was unable to fulfill the assignment. He actively labored as a missionary in the Eastern United States from 1840 until 1844, after which he rejected the leadership of Brigham Young and the Twelve and became associated with several rival factions. Page visited Nauvoo for a conference in April 1842, but then returned to Pittsburgh, where he resided until June 1843. While there he published a short-lived newspaper, the Gospel Light, and two pamphlets in refutation of the Spalding theory.³⁷

[I’m still waiting for evidence that Page ever published anything in the Times and Seasons.]

W. W. Phelps. W. W. Phelps had joined the Church in 1831 and been the editor of the Church’s first newspaper, the Evening and Morning Star, published in Independence, Missouri, from 1832 to 1834. Phelps had written several brief editorials discussing assorted reports of antiquities, including an article describing a ruined city in Central America. He left the Church during the troubles in Missouri in 1838 but returned and was rebaptized in Nauvoo during 1840. In 1843 he was considered, but passed over, for editor of the Nauvoo Neighbor, but there is evidence that Joseph Smith made use of him as a ghost writer for some material attributed to the Prophet during 1843 and 1844.³⁸ It is conceivable that [Page 37]he may have contributed to or authored some of the articles published during Joseph Smith’s tenure as editor.³⁹

[Brother Roper forgot to mention that Phelps contributed to the Times and Seasons in 1841. I think he also contributed in 1842. My analysis of the 900 words has led me to include Phelps as a co-author and/or editor, as spelled out in the Second Edition.]

Orson Pratt. Orson Pratt, also an early convert to the Church, was baptized in 1830 and became a well-known missionary and writer. Like his older brother and fellow apostle Parley, Orson labored in Great Britain from 1839 to 1841, after which he returned to Nauvoo with other members of his quorum. In May 1842, he was out of harmony with Joseph Smith and the Twelve over the issues relating to plural marriage. He returned to full fellowship in early 1843.⁴⁰

[“Out of harmony” in May? The relationship between his wife Sarah and John Bennett was a key factor in the Bennett scandals that plagued Nauvoo from July on. He went missing in July, possibly suicidal. In August, he was replaced as a member of the Twelve. Orson Pratt is one of the least likely plausible authors of the 900 words.]

Parley P. Pratt. Parley P. Pratt joined the Church in 1830 after reading and gaining a testimony of the Book of Mormon. His pamphlet A Voice of Warning was widely read; and an 1839 expanded revision cited several reports of antiquities from North and Central America which supported the Book of Mormon. He participated in the apostolic mission to Great Britain and from 1840 until 1842 was editor of the Latter-day Saints’ Millennial Star. Pratt returned to Nauvoo in early 1843.⁴¹

[I’m still waiting for evidence that Parley ever published anything anonymously in the Times and Seasons. However, it is possible he mailed the articles from England.]

William Smith. William Smith, the younger brother of Joseph Smith, was one of the earliest converts to the Church. In 1835 he was ordained an apostle and continued to serve in that office until the Prophet was killed in 1844. William’s relationship with Joseph and fellow apostles from 1835 to 1844 was sometimes contentious. In April 1842 he became editor of The Wasp in Nauvoo. In August he was elected a representative to the Illinois State Legislature, but continued to edit The Wasp until early December, after which he was replaced by John Taylor. Following the martyrdom, he became Church Patriarch, but in later 1845 he broke with the Twelve and was excommunicated; later he became associated with several religious factions.⁴²

[I think the evidence shows he was the actual editor of the Times and Seasons from about May through October 1. The Wasp and the Times and Seasons were printed in the same place and shared editorial content during this time frame.]

Erastus Snow. Erastus Snow was baptized in 1833. In the spring of 1840, at the suggestion of the Prophet he moved to Pennsylvania, where he served as a missionary in Philadelphia, New York, New Jersey, and [Page 38]Rhode Island. In September 1840 he returned briefly to Nauvoo to escort his wife back to Pennsylvania, where he returned the following month. In August 1841 he moved to Salem Massachusetts, where he labored until 1843. He briefly visited Philadelphia in April 1842, after which he returned to Salem, where he remained until his return to Nauvoo in March 1843. He would later serve as an apostle from 1849 until his death in 1888.⁴³

[I’m still waiting for evidence that Snow ever published anything on his own in the Times and Seasons.]

Charles B. Thompson. Charles B. Thompson joined the Church in 1835. After the Saints were expelled from Missouri, Thompson moved to New York. In 1841 he published his book Evidences in Proof of the Book of Mormon in Batavia, New York, and extracts from it were published in the Times and Seasons that same year.⁴⁴ He moved to Macedonia in Hancock County Illinois in the summer 1843. After the death of Joseph Smith he formed a Church of his own and led a somewhat colorful career.⁴⁵

[I’m still waiting for evidence that Thompson ever published anything in the Times and Seasons.]

Benjamin Winchester. Benjamin Winchester, who joined the Church in 1833, participated in Zion’s Camp in 1834. He published a newspaper, the Gospel Reflector, in Philadelphia from January 1841 to June 1841. From 1840 to 1843 he also published several books and pamphlets. Winchester was an industrious writer and missionary, but became a contentious figure during his time in Philadelphia from 1841 to 1843. He returned to Nauvoo in October 1841, where he was reproved by Church leaders for his conduct and counseled to do better. He briefly assisted as an editor of the Times and Seasons from November until January 1841, when the Twelve, at Joseph Smith’s direction, purchased the paper from Ebenezer Robinson. Winchester then returned to Philadelphia, where he continued to cause problems in the local branch. In June 1842 he again visited Nauvoo for a brief period, then returned again to Philadelphia until October of that year.[Page 39]He continued to cause difficulties in the Eastern branches of the Church until he was excommunicated from the Church in 1844.⁴⁶

[Brother Roper forgot to mention that Winchester had several anonymous articles published in the Times and Seasons in the 3^rd Edition of the Times and Seasons, that he had a long-time friendship with William Smith, that later in life he claimed he was at the Times and Seasons until John Taylor took over, etc.]

Although each of the above men had written on the Book of Mormon and pre-Columbian antiquities previous to 1844, Page, Snow, Thompson, and Winchester were not in Nauvoo during the fall of 1842, making them less likely candidates as writers of the unsigned editorials.

[There is no factual reason why their absence makes them “less likely.” Most of the material in the Times and Seasons was mailed in. The 15 Sep issue includes an advertisement for Winchester’s Concordance that had to have been mailed in, making Winchester the one known person who submitted something to the Times and Seasons for the 15 Sep issue.]

It is possible, however, that one of these men wrote the unsigned articles and with the help of a collaborator in the Nauvoo printing office may have succeeded in publishing them. Neville argues that Winchester may have done so with the assistance of William Smith while Joseph Smith was in hiding and unable to oversee the work in the printing office.

[No, not while he was in hiding and unable to oversee the office; the evidence suggests that Joseph had ceased acting as editor long before September.]

William himself may have written the unsigned editorials.

[Definitely possible, along with Phelps.]

The same could be said of George Adams, who met with Joseph Smith in September 1842 and, given his interest in the Book of Mormon, could conceivably have written or contributed to the editorials.

[This is implausible. Adams was an orator and actor, not a writer. Theoretically, anyone could write anything, but as a matter of practicality and probability, Adams is unlikely.]

Chronological considerations suggest that the Pratt brothers likely did not write them. Parley, though familiar with Stephens’ work, was in England in 1842. Unlike Parley, Orson was in Nauvoo in 1842, but was in the middle of perhaps the most severe emotional and spiritual crisis of his life. From May 1842 until January 1843 he was not involved in the work of the Twelve; and with his faith and marriage in crisis, American antiquities and Book of Mormon geography would likely have been the furthest topic from his mind.

As a journalist, Phelps could certainly write, was in Nauvoo at the time, and given his activities as a ghostwriter for Joseph Smith, should also be considered as a potential candidate. Although these candidates all seem less likely [they “seem” that way to Roper, but not to history] than Joseph Smith, John Taylor, or Wilford Woodruff, we have nevertheless included them in our statistical analysis below.

The comparison set thus has nine authors as shown in Table 4.[Page 40]

Adams, George J. Page, John D. Phelps, W. W.

Pratt, Orson Pratt, Parley P. Smith, William

Snow, Erastus Thompson, Charles D. Winchester, Benjamin

Table 4: Expanded Set of Comparison Authors. There are nine in total who can be considered plausible candidates due to possible historical connections.

[Notice who is missing? Joseph Smith, Wilford Woodruff, and John Taylor. Soon you will see why.]

How We Selected Texts Specific to the Style of the Central America Editorials: Efstathios Stamatatos, a specialist in textual analysis, says, “Any good evaluation corpus for authorship attribution should be controlled for genre and topic. … In addition, all the texts per author should be written in the same period to avoid style changes over time.”⁴⁷ To be able to distinguish clearly between authors, we focused on constructing a study with texts from the comparison group of authors that meet these three specifying criteria: (1) genre matched, (2) topic matched, and (3) time period matched to the unsigned Central America editorials.

Genre Matched: Since the Central America articles in question are editorials, for genre matching we selected only published works of an editorial or expository nature.

[Wait a minute. All the Joseph Smith data is holographic. Now Brother Roper is excluding holographic material from these alternative writers. So there’s a finger on the scale already. That’s why Joseph Smith is not included in this list, but this is your first red flag.]

This criterion is crucial because it is recognized that an author’s writing style can change with genre.⁴⁸ By focusing the text selection on the editorial or expository genre we did not include items such as personal letters, journal entries, or news items. If these other genres are included in the analysis set they can dilute the accurate characterization of the authors and confuse the results.

[Exactly! Brother Roper’s entire statistical analysis compares Joseph’s holographic writings with published articles. That is in direct contradiction to what he’s establishing here.]

Neville’s discussions refer to using a large corpus of articles and other writings — an apparent potpourri of genres; thus he subjects his conclusions to a multitude of potential confounding errors.

Topic Matched: The Central America editorials deal with parallels between the recently explored Central America ruins and the Book of Mormon. For topic matching, we selected only texts dealing with the relevant topic as indicated by key words or phrases from the Central America editorials, such as those shown in Table 5.[Page 41]

Antiquities*	Guatemala*	Narrow Neck of Land	Ruins
Central America	Incidents	Palenque	Stephens*
Darien	Isthmus	Quiriua	Zarahemla

Table 5. Typical Topic-Specifying Key Words in the Central America Editorials. The asterisks indicate that we included all spelling variations.

[This is even more fascinating. I’m eager to see which of Joseph Smith’s holographic writings include these terms. Except I know the answer: none of them. Nor are there any writing samples from John Taylor or Wilford Woodruff, apart from Woodruff’s journal that mentions Stephens. Now you see why these three “candidates” are not included in this part of the study.

Let me emphasize this. Brother Roper’s criteria here exclude the only three people he tested in his original article—and the first part of this article.]

These words are indicative of the topic covered by the unsigned Central America editorials. We did not use “Book of Mormon,” since that phrase is used many times in numerous articles that have nothing to do with Central America. Some phrases, like “authenticity of the Book of Mormon,” were not included, because they did not add any texts that were not already included by those in Table 5. Other words that seemed peculiar to these editorials were noted, but because they were not topic-specifying, they were not included.

[Okay, so before, context-specific words are not useful. According to Roper, “so-called “unique” marker words… have been shown to be nondistinguishing.” Now, only “topic-specific” words can even be included in the analysis! If words that “seem peculiar to these editorials” are in fact peculiar, then they are by definition not “commonly used by English speakers” which was Roper’s objection a few paragraphs ago.]

This topic criterion is crucial because it gives the best chance of matching authors’ styles with the Central America editorials.

[If the topic criterion is actually crucial, which of Joseph’s writings that Roper used fit this topic criterion? Which of Taylor’s or Woodruff’s, for that matter? The only time Woodruff mentions Stephens is in his journal, but Roper is now excluding journals. What we’re seeing here is Roper switching horses. He’s gone from a Joseph Smith corpus of holographic writings that never once mention Stephens or Central America, to a corpus by other potential authors that excludes everything holographic and everything that doesn’t mention Stephens or Central America.]  

The inclusion of other topics has the effect of producing a less focused style characterization.

[Except, apparently, if it is Joseph Smith. Then, the inclusion of other topics focuses style characterization. This is unbelievable.]

Neville includes a mix of topics in his textual analyses, thus adding further confusion to his results and diminishing the distinctiveness of the stylistic measures.

[I hope by now you can see the absurdity of this analysis.]

Time-Period Matched: The Central America editorials were published in 1842. For time-period matching we restricted the selected texts to those written from 1837 to 1852. [This is arbitrary and introduces bias. Some of these candidates stopped writing about this topic after 1844 or 1845. Others wrote about this topic before 1837. The Stephens books weren’t even published until 1841. How about if Brother Roper discloses his data and lets others play around with the dates?] Thus we excluded texts written in the 1880s, for example. This criterion is crucial because an author’s writing style can evolve over time.⁴⁹ [The article Brother Roper cites here compared writing styles of one author with a 30-40 year disparity and a second writer with a 27-year disparity. Here, Brother Roper applies a 15-year disparity.]

[Presumably, Roper’s corpus of Joseph Smith’s writings are also limited to 1837 to 1844; why not limit the others to that time frame?]

For example Sidney Rigdon’s writing style changed in his later years from his early years in the Church.⁵⁰ 

[I’m very interested to see what frontier authors didn’t change writing style over 15 years.]

Again, if too large a timeframe is included in an analysis, an author’s style in a relevant period can be diluted, and this can lead to inconsistent results. In his analyses, Neville includes references to texts from later time periods, which is methodologically unwise.

[I’m curious what Winchester texts are later than 1843. I don’t recall him writing anything after that, apart from a couple of letters. William Smith wrote some pertinent material through around 1846. I don’t recall using anything as late as 1852, much less later than that.]

Twenty-one texts comprising over 114,000 words from the expanded set of authors were found to match these three important specifying criteria.⁵¹ 

[Brother Roper’s footnote says “A list of all of the texts used in the analyses is available from the authors to interested researchers upon request.” I have been requesting a list of his texts since January 2015, but so far he has declined to provide it. I’ll consider this my latest public request. It would not have been difficult for Brother Roper to list the texts here, or at least to provide a link. Instead, he insists on screening the recipients of his data—a classic attribute of Black Box scholarship.]

Note that we were careful to consider all the Winchester texts[Page 42]mentioned by Neville in his Appendices II and III. Most are off topic. Those that are on topic were included in the analyses.

By focusing on the published editorial or expository genre, the Central America ruins topic, and the relevant timeframe, we compiled a set of texts that can specifically distinguish between the authors relative to the Central America editorials. Without meeting these crucial criteria, analyses can give misleading, erroneous results. Significantly, Neville’s naïve analyses do not meet any of these criteria.

[Hmm, so Brother Roper has excluded Winchester’s March Gospel Reflector articles. Well, let’s see how many of Brother Roper’s Smith, Woodruff and Taylor texts meet the criteria. I have a sense this sentence is going to be the summary of my review of this article: “Without meeting these crucial criteria, analyses can give misleading, erroneous results.”]

How We Prepared the Texts for Analysis: To guarantee that each and every word was correct, we independently verified our electronic texts against photo copies of the original publications. If this is not done, the computed frequencies of word usage can misrepresent each author due to typographical errors. Neville did not verify all of the texts he used in his analyses.

[Actually, I pointed out that the electronic texts have errors, but so do different editions of the published texts. Every edition of Voice of Warning has significant changes, for example.]

To get an unsullied characterization of the authors, we also deleted all non-authorial words, like quoted material and scriptural references. If this is not done, an author’s words can be mixed with the words of other people and can once again lead to mischaracterization of his or her style. Neville did not make this effort consistently in his analyses.

[Actually, I did, but oh well. But I think frequent quotations of similar scriptural material is relevant.]

How We Found Truly Distinguishing Words: Since all the authors share many words in common, it is necessary to find which words are truly distinguishing. A criteria-based method of selecting words can be used to provide a sound, unbiased basis for decisions. To obtain a set of truly distinguishing words, we examined all words; only those that met the following four criteria were selected:

1.     The word has to be a noncontextual word. This is a standard approach in stylometry, as discussed previously. [So we’re using contextual words to choose the database, and then we’re excluding them.]

2.     To help guarantee that the words used will differentiate among authors, the word has to be one of the words whose range of proportions is in the top five percent of all the words. A lower percentage gives too few words; a higher percentage gives too many. [How can this determination be made before we run the analysis? Let’s see the data on this.]

3.     To guarantee that the word is used frequently enough to give statistically meaningful results, the overall pooled proportion for the word has to be greater than one in a thousand. [There aren’t even 1,000 words in the composite of the unsigned editorials.]

4.     To help guarantee statistical relevancy and ensure that the word is characteristic of the author of the Central America editorials, the word must appear at least three times in the composite Central America editorial texts. [Are we talking about the 900 words or the five articles? As I’ve pointed out, Brother Roper simply assumes all three (or five) editorials were written by the same person. In fact, all are anonymous; they could have been written by the same or different people. Plus, they all could have been edited. At an average of 300 words each, it is questionable whether these articles are even susceptible to statistical analysis. If a given word must appear at least three times to “help guarantee statistical relevancy,” and the word doesn’t appear at least three times in any one of the articles, then that word, by Brother Roper’s own criteria, is invalid. Consequently, Brother Roper’s own analysis can’t work except on his arbitrary assumption that one person wrote all there articles! Not only that, the articles use the editorial we, which could be the editor’s (whom we know was not Joseph Smith) or a combination of authors, or an editor and author jointly.]

[Page 43]Thirty-seven noncontextual words met all four criteria: a, all, and, are, as, at, be, been, but, by, can, could, from, has, have, his, in, is, it, more, not, of, on, our, so, such, that, the, they, this, those, to, upon, was, we, will, with. Selecting words in this fashion helps distinguish among authors using statistically significant words specific to the Central America editorials. This methodological rigor contributes to achieving the goal of high overall specificity for the study.

Validation of Word Selection Method and Discriminant Analysis: To demonstrate the usefulness of our criteria-based word selection method and discriminant analysis, we applied them to The Federalist Papers. This yields seventy-five noncontextual distinguishing words. Figure 7 shows discriminant analysis results for these words.

[This is a preposterously irrelevant comparison. None of the Federalist Papers is as short as 300 words; there are only three possible authors; most of the Papers were claimed by their authors; all the papers dealt with the same subject matter, and so forth. Brother Roper here is invoking the gold standard of stylometry to confer illusory legitimacy on his own analysis.]

The discriminant analysis had 99% correct classification of the seventy training-set papers. Three papers are considered co-authored by Madison and Hamilton, so they were not included. All but one of the disputed papers is assigned to Madison — the attribution generally accepted by historians.

Figure 7: Discriminant Analysis for The Federalist Papers. There are clear separations among authors, and all but one of the disputed papers are assigned to Madison, consistent with the findings of previous historical and stylometric analyses.

[Page 44]Will this method also discriminate between The Federalist Papers authors and Winchester and Neville? Including Winchester and Neville as negative controls in the discriminant analysis generates the plot shown in Figure 8.

As they should, Winchester and Neville clearly separate from Hamilton, Madison, and Jay, as well as from the disputed papers. Interestingly, Neville’s style is the most distinctive, as captured in the first discriminate function (horizontal axis). Winchester’s style, the next most distinctive, is contrasted with Jay’s style in the second function. Although not shown in Figure 8, the third function displays less separation between Hamilton and Madison, who are known to be similar in style. Thus we can see that the criteria-based, word-selection method, coupled with discriminant analysis, form a powerful and accurate technique.

[Of course, Brother Roper didn’t apply his exclusionary criteria to this analysis, which makes one wonder what was the point of all the discussion about genre, topic and time frame.]

Figure 8: Discriminant Analysis of The Federalist Papers, including Winchester and Neville. Winchester and Neville are easily distinguishable from The Federalist Papers authors.

Objective, Scientific Hypothesis Test Methodology

Having observed that the three Central America editorials are unsigned and that Neville offers Winchester as the author, we formulated the research question as follows:[Page 45]

Is the writing style in the Central America editorials closer to Benjamin Winchester than to the other candidate authors in the expanded set?

To test this research question we formulate the null (H₀) and alternative (H_a) hypotheses as follows:

H₀: Winchester’s style is not the closest to the style of the Central America editorials among the other comparison authors (at least one other is closer).

H_a: Winchester’s style is the closest to the style of the Central America editorials among the other comparison authors.

Note that since we have already shown the results of an analysis comparing Winchester with Joseph Smith, John Taylor and Wilford Woodruff, in this analysis we compared only Winchester to the other comparison authors.

[Notice, Brother Roper can’t apply Joseph Smith, John Taylor, or Wilford Woodruff here because they don’t satisfy the qualifying criteria. So the first analysis is unreliable because of the secret data problems and the second is unreliable because we still have secret data problems.]

Performing discriminant analysis, we obtained the plot of group centroids shown in Figure 9. Six comparison authors are closer to the Central America editorials than to Winchester. The probability that the Central America editorials belong with the Winchester texts is less than one in a thousand (< 0.001).

Figure 9: Group Centroids from Discriminant Analysis. Winchester is not the closest to the Central America editorials. Neither is William Smith. We point out William Smith because Neville conjectures he could have been another possible source of the editorials.

[Page 46]Robust Results: Many studies rely on only one approach to analyze the styles of authors. But in order to not be fooled by the results of only a single approach, we incorporated an array of analysis techniques to confirm that the results are consistent and reliable.

[This is another transparent garbage in, garbage out problem. It’s the choice and manipulation of the data that is relevant, not the statistical manipulation of the data.]

When viewing the data from these various angles we can see a more robust picture of the real situation. With the word-use proportions for the selected words for each author, we performed the following analyses: Burrow’s Delta Method, Discriminant Analysis, Fisher’s Combined Probability Test, n-Gram Matching, and Principal Components Analysis.⁵²,⁵³

To help ensure that the results were not affected by the number of texts we included for each author, we checked to see if there is any relationship with sample size. There was no evidence of a strong relationship. This indicated that the stylometric results are unaffected by varying sample sizes.

[Seriously? We’re supposed to take their word for all of this? When they have admitted all along that their objective was to show Winchester was not the author?]

This array of five analytic techniques showed Winchester to be an even worse candidate among a group of other plausible candidates for authorship of the Central America editorials than when he was compared to Joseph Smith, John Taylor, and Wilford Woodruff. The results are shown in Table 6.

[I don’t have any reason to question the statistical analysis here, as a function of math. I do question the methodology associated with the data, and my questions, so far, remain unanswered.]

Test Method	Number of Comparison Authors Closer than Winchester to the Editorials
Principal Components	8
Discriminant Analysis	6
Burrow’s Delta	6
Fisher’s Method	2
n-Grams Matching	2

Table 6: Number of Comparison Authors Closer than Winchester to the Central America Editorials. Among the expanded set of comparison authors, for each test there are always other candidates who are closer to the Central America editorials than Winchester.

Winchester is never the closest in any of these tests: Two to eight other candidates are always closer in style to the Central America editorials than Winchester. The highest he ever ranked was a distant third place. Consequently, once again we find no persuasive evidence that [Page 47]Winchester is a good candidate for authorship of the three unsigned Central America editorials.

[This outcome was predictable from the title of the article, so it’s no surprise. But we still have no idea what Roper has done here with his secret data.]

Stylometric Evidence Conclusion: The results of multiple formal, statistical tests of hypothesis combined provide consistent, overwhelming lack of evidence that Winchester is a viable candidate for authorship of the unsigned Central America editorials.

[This is the same black box we’ve been seeing all along.]

Neville’s Highly Speculative Style

In contrast to the evidence provided by these objective tests, Neville’s conclusions throughout his book are not based on facts, but on a continual framework of conjectures, speculations and suppositions — so much so that they can be easily measured. He frequently uses speculative words such as could, maybe, perhaps, possibly, seems, suggests, supposedly, and a host of other similar words. Figure 10 shows a “word cloud” to illustrate how frequently he uses speculative words. The most prominent word is suggests.

[This is exactly what one does when one searches for the truth. One takes the historical data—and I evaluated all the historical data, not merely whatever matched my preconceptions the way Brother Roper does—and proposed alternative explanations. Anyone can see the facts I’ve assembled and can reach their own conclusions. That’s all I’ve done. In my view, my proposal best fits all the historical data, including the personalities and motivations of the people involved.

By contrast, Brother Roper’s proposal that Joseph Smith wrote these 900 words contradicts the historical data and what we know of the personalities and motivations of the people involved.

All I do is present the data. Anyone is free to reach the same or different conclusions.

What I object to is Brother Roper’s long-established pattern of ignoring facts that contradict his theories, inventing facts he can’t establish (such as everything he thinks Joseph did just happened to be omitted from Joseph’s journals and all extant letters, documents, and journals kept by others). Brother Roper’s theory requires not only that we leave known facts unexplained, but that we assume a long list of facts not in evidence that contradict known facts. My theory explains every known fact and makes inferences about the unknown that are consistent with what is known.

In this article, Brother Roper has worked hard to come up with statistical analysis that supports his theory. To do so, he has used a corpus of Joseph Smith’s writings chosen by a set of criteria that he later completely disqualifies as unreliable for other candidates. Then he acknowledges that he is keeping his database secret.]

To see how unusually often he uses speculative words, we compared them to the frequencies tabulated in the Corpus of Contemporary American English (COCA), available online at http://corpus.byu.edu/. It is described as follows.

COCA is the largest freely-available corpus of English, and the only large and balanced corpus of American English. The corpus was created by Mark Davies of Brigham Young University, and it is used by tens of thousands of users every month — linguists, teachers, translators, and other researchers. The corpus contains more than 450 million words of text and is equally divided among spoken, fiction, popular magazines, newspapers, and academic texts. It includes 20 million words each year from 1990‒2012 and the corpus is also updated regularly. The most recent texts are from summer 2012. Because of its design, it is perhaps the only corpus of English that is suitable for looking at current, ongoing changes in the language.[Page 48]

Figure 10: Word Cloud of Neville’s Speculative Words. The font size of each word is in proportion to how frequently Neville uses that word in excess of common usage in American English today.

Using COCA, we calculated the difference in relative frequency of Neville’s use of speculative words compared to their relative frequency in standard American English today. Figure 11 shows the ten words with the largest differences.

Figure 11: Neville’s Top Ten Speculative Words Compared to the Corpus of Contemporary American English. Neville uses speculative words more frequently than standard American English.

We can see that Neville uses these words in higher frequencies than commonly used in Standard American English. For example, he uses suggests almost 1,000 times per million words more often in The Lost City of Zarahemla than people use the word on average in a wide spectrum of texts.

[Seriously? How about comparing my word usage to other historians? By its nature, history requires us to draw inferences from limited facts. We make those inferences and offer suggestions. IIRC, even a few archaeologists use these terms… (See Mormon’s Codex, which uses these terms throughout.)

So I’ll embrace this criticism. I’ve been explicit in my writing about what is factual and what is an inference.

Earlier on in his schizophrenic review, Brother Roper complained that I claimed everything was a fact. Now he complains because I don’t. 

I offer suggestions and plausible explanations. Readers can easily see what is factual in my book; I use footnotes extensively. By contrast, throughout this paper, Brother Roper hides facts in favor of misleading his readers.

Throughout my book, I invite additional research and perspectives. I propose alternative interpretations of the data and welcome anyone’s opinion (although I prefer that people rely on actual facts, instead of imagining what might have been, heedless of the facts, the way Brother Roper consistently does.)

My approach is in stark contrast to what we find in Brother Roper’s work, such as this review. Instead of inviting additional research and analysis, he refuses to disclose his data, methodology, and assumptions. Instead of even acknowledging the many weaknesses of his approach, he claims infallibility and certainty.

That’s language of confirmation bias, not legitimate research.]

Figure 12 shows cumulatively how frequently Neville uses speculative wording in ten-page increments in the first 192 pages of his book. In summary, Neville uses speculative wording over 800 times in the first 192 pages of his book. In one ten-page segment he uses an average of almost nine speculative words per page. We can see that he starts off using speculative words at a high rate, and then his rate of using [Page 49]speculative vocabulary increases as his narration continues. From the information displayed in Figures 10, 11 and 12, we can describe Neville’s style in The Lost City of Zarahemla as “highly speculative.”

Figure 12: Frequency of Neville’s Speculations. In the first 192 pages of his book, Neville uses speculative words over 800 times, and his speculation rate even increases as he goes along.

Neville’s speculative language indicates the nonresearch nature of his work, since speculative language is used more frequently in popular articles than in research articles.⁵⁴ 

[I trust readers to make up their own minds based on the evidence I present. My approach is the antithesis of Black Box scholarship.

Brother Roper, by contrast, tells readers what he considers “fact” without giving them the underlying data to make up their own minds. His results are not replicable and his data is unknowable; mine can be accessed by anyone.

I’ll let you decide which approach is more credible.]

Two linguists who have studied speculative language and its functions, Elsa Pic and Grégory Furmaniak, state, “If such hypotheses [speculations] were too numerous in research articles, they would be severely received, as readers of [research articles] are peers who do not accept unsupported conjectures and do not expect to be treated as less knowledgeable.”⁵⁵

Neville’s Speculations Unscientifically Morph into Facts

Within a cloud of speculation, Neville is unable to distinguish fact from fiction. He accuses Winchester of creating facts out of the whole-cloth of inference (123) and disparages “Winchester’s inference … which morphed into a fact in his Times and Seasons articles” (p. 180). Yet he himself does the same thing.

[Well, the 15 Sept. article in the Times and Seasons does assert as fact something that is patently untrue. Roper wants us to believe it was Joseph Smith who lied about the Book of Mormon. I think Joseph knew the text better than that.]

Neville repeatedly creates “facts” morphed out of the whole-cloth of his own original inferences, suppositions and speculations. For example, on page 7 he speculates: “led me … to suspect someone else entirely [Page 50]had written the 900 words.” Then one page later he asserts flat out, with no hedging: “Joseph did not write these editorials.” Throughout his book, there are numerous such morphs of speculated conjectures into statements of fact.

Neville asserts as facts his speculations and spins a tale based merely on things he imagines seeing in the data. If we use his speculative vocabulary, The Lost City of Zarahemla “suggests, perhaps, that maybe, it appears, that it could be that” his imagination is reality.

[Anyone can read the historical facts I’ve cited and see for themselves. It’s an open model that invites further research and analysis. In his reviews, Brother Roper hasn’t contested a single historical fact I’ve cited—but he has ignored dozens, if not hundreds. Instead, he insists Joseph did and wrote things for which there is not only no record, but there is no precedent or antecedent. Everything Brother Roper asserts about Joseph Smith’s authorship of these articles depends on the absence of evidence—things not recorded in journals, letters, or other contemporary accounts, things written anonymously, things written using words and phrases Joseph never used at any time before or after, and so forth. Then, in a last-ditch effort to prove with statistics what he can’t even plausible establish with historical facts, he applies two phony stylometry analyses that, in the first instance, applies special rules to Joseph Smith’s writings, and in the second instance, excludes Joseph as even a possible author!]

Conclusion

Our previous article, “Joseph Smith, The Times and Seasons, and Central American Ruins,” concludes that our analysis pointed to Joseph Smith as the most likely author of the Central America editorials, with possible influence from John Taylor and Wilford Woodruff.

In The Lost City of Zarahemla Neville conjectures that there is another author — Benjamin Winchester — and spins an elaborate tale based on speculation and imagination which he often states as fact while weaving a baseless story of conspiracy.

[It’s baseless only because Brother Roper doesn’t engage the historical facts. I included hundreds of footnotes, mostly to original sources. By contrast, Brother Roper offers 7 historical facts, all of which are erroneous. I trust readers to decide whose theories are baseless. Brother Roper compounds his misleading history with even more misleading “black-box” statistical analysis of “black-box” data.]

He ignores the simple fact that unsigned editorials are common in newspapers then and now, and do not imply a clandestine desire for anonymity by the author. [Wow. It would be helpful if Brother Roper would do some research on this point instead of making more bald assertions, but his main citation in this paper contradicts what he’s saying here. The Federalist Papers were all signed by a single pseudonym. The authors didn’t take credit until much later in life. Brother Roper would have us believe they did this because… why, exactly? Because they wanted everyone to know who they were?

While that idea makes as much sense as most of Brother Roper’s arguments, I suspect not even he believes that. There is a long history, particularly in America, of desires for anonymity. In this specific case, of all the potential authors for the 900 words, only Winchester had been formally silenced for all the Church to see. Who of the candidates had a reason to remain anonymous other than Winchester? That’s a question Brother Roper doesn’t even attempt to answer.]

In fact, the most logical assumption then and now is that the editor is the author of unsigned editorials.

[This assumption is neither logical nor factual. Most of the unsigned material in the Times and Seasons consists of reprints from other periodicals with the author’s name omitted, such as the many articles from Winchester’s Gospel Reflector. However, if Brother Roper’s statement here is a concession that William Smith wrote the articles, I could accept that as a starting point. Or maybe Brother Roper is back to insisting that Joseph Smith sent himself a letter to publish, pretending he was someone else?]

It is also common practice now, as it was then, for editorials to be the “voice” of the editor expressing the opinion of the publisher.

[This could use a citation or two.]

The most logical assumption is that editorials — signed or unsigned — are official statements of the people responsible for the newspaper.

[This might be actually be a rational argument—the first one so far Brother Roper has made—but it still begs the question of who was responsible, not for the paper, but for writing and publishing these 900 words. The facts show Joseph was not directly involved and probably didn’t go to the printing office more than once a month. Of course, Brother Roper doesn’t address the facts; to him, what the historical record does not say is more important than what it does say. So it’s no wonder he relies on “logical assumptions” instead of facts, but the lack of supporting facts render this argument invalid, after all.]

When that is not the case, a disclaimer is published which says, in effect, “The views expressed in this article are not necessarily the views and opinions of the editor or publisher of this newspaper.”

[I’m curious about which frontier newspaper did this. Brother Roper’s claim here is an anachronism comparable to the claim that the Stephens ruins are actually Zarahemla.]

Further, he ignores the fact that it is completely irrational for Joseph Smith to have published in the Times and Seasons three editorials of unknown authorship that contradicted his views, since he took over the editorship due to his concerns for what was being published in the paper. And it would be even more irrational for him to publish material for which he did not know the author after he had assured his readers at the onset of his editorship that he was responsible for the content of the paper with the clear statement “I stand for it.”

[This argument has been so completely debunked I won’t repeat that again here.]

Even in the unlikely event that something he disagreed with had slipped by his notice and was published three times, Joseph Smith still had numerous opportunities and venues to correct those statements, [Page 51]even after he was editor. There is simply no logical basis for Neville’s characterization of the publication of the unsigned Central America editorials as being contrary to Joseph Smith’s views and due to clandestine conspiracy. The Prophet could have corrected any errors at any time.

[I address this at length in the book. Joseph did take action. Furthermore, the Zarahemla in Quirigua was never repeated. After Joseph died, it was superseded by Zarahemla in South America.]

The first article in a series of three Interpreter articles showed that Winchester did not promote a limited Mesoamerican geographical setting for the Book of Mormon, but rather a hemispheric one. His ideas were nothing new and thus did not warrant any subterfuge for their dissemination.

[If the 900 words were nothing new, why did Brother Roper write the first stylometry article? Why did he write 55,000 words trying to attack my summary and analysis of the history?  I give him credit for effort, but ultimately the desperation inherent in his Black Box scholarship and inability to address the historical facts demonstrates that his theory of what happened just doesn’t work.]

The second article showed that Joseph Smith was not opposed to considering Central American cultural, geographical, and historical correspondences with the Book of Mormon, but to the contrary found them interesting and supportive of the Book of Mormon.

[The only historical evidence Brother Roper has provided of Joseph’s “interest” in Stephens and Central America is the Bernhisel note, and that is far more questionable than Brother Roper acknowledges. Otherwise, Brother Roper’s arguments are a pile of inferences and suppositions, compiled into what he thinks are “logical” conclusions. By contrast, I’ve marshalled abundant historical evidence to support my proposals.]

In this third article we have shown the inadequacy of Neville’s arguments. Neville says he sees an “outlier” in the discriminant plots in our article “Joseph Smith, the Times and Seasons, and Central American Ruins.” But statistical tests contradict his “eyeball” test and show no evidence that the Central America editorials are inconsistent in style with the texts in the Joseph Smith Group of texts. So the foundational premise for his book is false. What he sees is due to his preconceived bias for Winchester’s authorship of the unsigned Central America editorials.

[This is a funny Freudian slip by Brother Roper. It was impossible for me to have had any “preconceived bias” because I had never heard of Winchester until I did the research. I simply followed the evidence where it led.

By contrast, Brother Roper has made a career out of promoting the Mesoamerican theory. The secrecy and inconsistency of the stylometry approaches he describes in this article reflect “preconceived bias” of the highest order.  I’m more than willing to undertake a full, open, objective examination of the stylometry evidence if/when Brother Roper is equally willing to do so. So far, he’s refused. That says more than anything this article purports to say.]

We have also shown that Winchester is no better candidate than Joseph Smith as author of the Central American editorials; and we have further shown, using an array of objective statistical techniques, that Winchester is a poor choice among an expanded set of comparison authors. The historical and stylometric evidence is overwhelmingly against Winchester as the author of the Central America editorials.

[Brother Roper’s “expanded set of comparison authors” excludes Joseph Smith, Wilford Woodruff, and John Taylor from consideration. By definition, that makes Winchester a more likely candidate than any of them. I’m interested to see the underlying data. We’ll see what Brother Roper is willing to share in the future.]

Neville’s book is at best a work of fiction. In fiction an author can create an imaginary world to match the way he or she wants things to be. However, in history and science we are constrained by the evidence provided by data. There is only imagination in Neville’s pseudo-science masquerading as history. The Lost City of Zarahemla is just the latest entry Neville has added to the list of his other novels.

[There is admirable irony in this conclusion. Brother Roper’s entire historical analysis consists of imagining what Joseph Smith could have done in his spare time, unrecorded, and unremarked upon. His theory contradicts everything else Joseph wrote or spoke about, as well as everything his contemporaries wrote or spoke about what Joseph said and did. Brother Roper’s theory ignores the real-world situation in the printing office in 1842, the personalities involved, and the motivations of the people. Even historical novels are based on some facts. In that sense, Brother Roper’s theory can’t even qualify as a historical novel; it’s pure fantasy.]

Appendix:
Dissection of Neville’s Pseudo-Stylometric Statements in Appendix III of The Lost City of Zarahemla

By his own admission, Neville is an amateur when it comes to stylometry (p. 219). Since he evinces no experience, expertise or sound judgment in stylometric research, it is not surprising that he uses archaic, low-power,[Page 52]nondistinguishing methods, and jumps to baseless conclusions. In the following we address by topic each of his assertions in Appendix III (pp. 217‒33) of The Lost City of Zarahemla.

[That’s a nice string of adjectives, but it contradicts what I actually wrote in the book. My skepticism of stylometry has been reinforced by the inconsistent methods described in Brother Roper’s paper.]

Excessive Variation

• Neville says that a problem in applying stylometric analysis is that the unsigned Times and Seasons editorials vary so widely in style, content and approach that they cannot be grouped to Joseph Smith (pp. 217‒218).

>> This statement is unfounded. Discriminant analysis shows that the unsigned editorials group together and cluster with Joseph Smith’s writings and editorials.

[Well, the analysis described in Brother Roper’s papers claims the five articles had to be grouped together to have sufficient data for analysis. That constitutes assuming the outcome; i.e., Brother Roper simply assumed the same author wrote each one, and then sought to prove his thesis that Joseph was the author by using different criteria for the Joseph corpus than he used for everyone else.]

Outlier Claims

• Neville says that we have previously concluded that Joseph Smith is the author “because his writing style is a little closer to the unsigned articles than are the styles of Wilford Woodruff and John Taylor” (p. 218).

• He claims that these previous studies by Roper and Lund tend to show that Joseph Smith is not the author of the unsigned editorials (p. 218).

• He repeatedly asserts that the Central America editorials are “so distant from Joseph that it appears to be an outlier” (p. 219) and that the composite of the Central America editorials “appears to be an outlier” (p. 220).

>> Univariate and multivariate distance measures show that the Central America editorials are much closer to Joseph Smith than to John Taylor or Wilford Woodruff. Multiple analyses testing for extreme values show that the Central America editorials are not incompatible with the Joseph Smith Group of texts. Neville’s opinions are not supported by objective statistical analyses.

[Continually repeating the same mantra without addressing the fundamental problems is not persuasive. The statistical analysis is not the issue; it’s the validity of the data and the assumptions. We have five separate articles, all by unknown authors. There is no principle of stylometry that justifies combining them just because the researcher wants to have a single author for each of them, or just because the researcher deems the articles, individually, as too short for a legitimate analysis. No principle of stylometry legitimizes the testing of one author’s holographic material, in many cases over a decade old, involving topics and genres entirely different from the other proposed authors’ corpus, against published and edited work by the other authors and a composite of published and edited short pieces.]

“Someone Else Wrote Them”

• Neville says, “In my layman’s opinion, Roper’s results suggest someone other than the three writers he tested actually wrote the 900 words” (p. 219).

• He asserts that analyses by Roper and by Lund “assume” that the only possible authors are Joseph Smith, Wilford [Page 53]Woodruff, and John Taylor; he ignores the role of William Smith (p. 218).

• He claims that Roper made a “simple mistake, … forgot about The Wasp,” i.e., about The Wasp’s editor, William Smith (p. 220). [So supposedly I “ignored” the role of William Smith but I focused on him? Nowhere did Brother Roper consider William Smith in his original paper.]

>> Joseph Smith, John Taylor and Wilford Woodruff are the three candidates for whom the historical evidence is the strongest, since they were responsible for the paper and were known to be directly connected with the Times and Seasons production during this time.

[But in his more detailed stylometry analysis, Brother Roper excludes them as possible candidates! They don’t pass the genre, topic, or time criteria.

Plus, most of the material published in the Times and Seasons was mailed in. Local authors were identified by name (Eliza Snow, Phelps, Littlefield, and even Woodruff and Joseph Smith when they actually wrote articles). There is no historical evidence to support the idea that Joseph, John, and Wilford were candidates for anonymous articles, let alone the strongest candidates. To the contrary; the historical evidence excludes all three as viable candidates (short of assuming they wrote these articles months in advance, a proposition that the articles themselves refute.)]

All other candidates are only circumstantially possibilities. The Extended Nearest Shrunken Centroid Method (ENSCM) open-set test found no evidence of a latent author, and thus no need to consider another candidate besides the historically justifiable Joseph Smith, John Taylor and Wilford Woodruff.

[This is nonsense. By Brother Roper’s own criteria, there is no corpus for any of these three men that qualifies for analysis.]

Even so, when Benjamin Winchester and William Smith are included individually as possible candidates along with Joseph Smith, John Taylor and Wilford Woodruff, and when they are tested as part of the extended set of comparison authors, statistical tests show repeated that neither is a likely candidate.

[This is the sleight-of-hand trick. Joseph, John, and Wilford were not—the could not have been—tested along with the extended set of comparison authors. The test of Winchester against Joseph, John, and Wilford used a different analysis than the original analysis, and the results were not fully disclosed even then. Needless to say, Brother Roper never disclosed his data, parameters, etc.]

In fact, when we took each author from the extended set of authors (Table 4) and used discriminant analysis to compare his writings to those of Joseph Smith, John Taylor and Wilford Woodruff, for nine out of nine comparisons, the Central America editorials are closer in style to Joseph Smith than the comparison author, and the lowest probability of group membership for the editorials in the Joseph Smith Group is 99%. We did not forget anyone. No one else is a more likely candidate than Joseph Smith. There is no evidence that “someone else wrote them.”

[This is nothing but Black Box garbage in, garbage out.]

About Techniques

• Neville says, “A writer’s use of function words can be unique enough to yield statistically significant results” (p. 217).

• He claims that collocation habits and rare pairs can be distinguishing (p. 217).

• He claims, “I decided to apply similar stylometric methodology” (p. 218).

>> Though Neville recognizes the value of function words[Page 54](noncontextual words), he does not use them in his analyses. We use them in our analyses. Further, an author’s word pattern habits can be distinguishing, yet Neville analyzed only a few collocation and word pairs, whereas we analyzed all the phrases from two-word to six-word sequences for the extended set of candidate authors in comparison to the Central American editorials. We looked for the author with the greatest number of phrases in common with the Central America editorials. As Figure 13 shows, Winchester was never the top choice. The closest he ever came was third place — at best a bronze medal but never a gold medal.

Figure 13: Comparison Authors with More Two-word to Six-word Phrases in Common with the Central America Editorials than Winchester. Winchester always shares fewer phrases with the editorials than do other authors.

Neville did not use valid and reliable stylometric techniques, so his claim of applying “similar stylometric methodology” is a gross misrepresentation.

[It bears repeating that Brother Roper excluded Joseph, John, and Wilford from his extended analysis.]

Average Sentence Length

• Neville uses average sentence length (ASL) as an authorial style metric (pp. 217, 225).

>> ASL is particularly weak and nondistinguishing as a measure of authorial style. It is an antiquated and amateurish metric. The following shows the deficiency of ASL as a stylometric measure:

Splitting each of the comparison author’s composite texts into blocks that are roughly the size of the Central America editorials while maintaining whole sentences, the ASLs of [Page 55]the blocks are not consistent within an author, thus violating a crucial criterion of a useful stylistic measure. The ASL of the Central America editorials is 32. Figure 14 shows the range of ASLs of the comparison authors and of the ASL of the Central America editorials.

Figure 14: Ranges of Average Sentence Length (ASL) of Expanded Set of Comparison Authors and the ASL of the Central America Editorials. The Central America editorials’ ASL is within the range of ASLs for all comparison authors except William Smith. Clearly, ASL is not a distinguishing measure.

Winchester’s ASL is not distinguished from the other comparison authors’ ASLs. The range of ASLs for each comparison author overlaps Winchester’s ASL range. All the comparison authors’ ranges overlap each other. The ASL for the Central America editorials is within the range of all the comparison authors’ ASLs, except William Smith’s (who by Neville-logic would thus be disqualified as the author). ASL is obviously a weak and nondistinguishing measure.

Skilled stylometricians abandoned using ASL a century ago. Neville should also.

[This is an illusory complaint. I was responding to Brother Lund, on his terms.]

Unique Words

• Neville discusses “unique” words or phrases he claims are “exclusive to one author” (pp. 222‒25).

>> This is not a distinguishing metric, as we have shown with The Federalist Papers example.

[Except the experts Brother Roper himself cited claimed the use of “marker” words on the Federalist Papers was effective. Brother Roper’s academic citations in the paper contradict his own assertions in more than just this instance.]

In addition, each of the candidate authors in the extended comparison group has so-called unique words compared to those of others. These range from 13% to 27% of their words, [Page 56]with Winchester having 17% “unique” words. Seven of the other authors have more of their unique words appearing in the Central America editorials than does Winchester. Using Neville’s unique-words approach would actually disqualify Winchester as the author of the unsigned editorials, since other authors are better choices based on so-called unique words. Even Neville himself has some of his unique words in common with the unsigned Central America editorials and, in fact, more “unique” words than Winchester. Using Neville-logic, this is evidence that he wrote the editorials. By his own method, Neville is a better choice for author of the editorials than Winchester.

[Hopefully any reader can see how ridiculous and unscholarly this paragraph is.]

Similarities

• Neville cites his Appendix II, where he annotated words in the Central America editorials and notes that Winchester also used these words (p. 218).

• In particular, he focuses on three words: foregoing, credulous, andincontrovertible; and points out that Winchester also used these words (pp. 221‒22).

• He discusses phrases offered by Lund such as assist us to, cannot doubt, cuts, the eyes of all the people, and so forth. (p. 223).

• He focuses on several more phrases and words: none can hinder, so much, surely and great joy (pp. 225-26).

>> Neville bases much of his “analysis” on “cherry-picking” similar wording and uses them to imply equality of source (same authorship). His approach of searching for similarities is nothing more than snooping around in the data looking for confirmatory evidence.

[Again, this was a response to Brother Lund’s approach.]

To see how absurd and misleading this can be, we applied his method to his own book and looked for similarities between Winchester and Neville. We found over fifty examples. Using Neville-logic, these similarities between Neville and Winchester would mean that Neville and Winchester are the same person, but such a conclusion is obviously absurd. To Neville, these similarities would be crucial “facts” that prove equality, but such reasoning is vacuous and intellectually dishonest.

[This is an outlandish claim, of course. The key in all of this, as Brother Roper’s schizophrenic article admits at some points but disputes at others, is historical context. I framed this as means, motive and opportunity. The lack of these exclude most of the universe. Once all three are established, then one can look at similarities. Here, Brother Roper’s irrational and inconsistent stylometry analysis insists on Joseph Smith writing 900 words about Central America and Stephens and the like for the first and only time in his entire life, in a historical setting that makes it almost impossible for him to have done so. Then Brother Roper excludes the one person who wrote specifically on these topics and had the means, motive and opportunity to write the 900 words.

Brother Roper is so befuddled that he excludes Joseph as a possible candidate for one of his own stylometry tests. So it’s not only the historical context and authorial content that exclude Joseph: it’s Brother Roper’s own criteria!

Nevertheless, Brother Roper insists that only Joseph could be the author.]

[Page 57]Although we can find similarities between two things or people, similarity does not establish sameness or equality. Neville commits the fallacy of equating Winchester with the author of the Central America editorials because of “similarities” he thinks he sees. It is always possible to find any number of superfluous similarities between two things, if we are determined enough, but similarity does not establish equality.⁵⁶

[Nice! A long-awaited reference to the Interpreter’s citation cartel—even if it is to support a mere aphorism.

But think about this. Brother Roper can find zero similarities between Joseph Smith’s entire writing corpus and the 900 words. Instead, he manipulates both the corpus and the 900 words (actually, the five articles) until he can find a discriminant that satisfies his desired outcome. But then he’s caught in a bind because he realizes Joseph Smith doesn’t even qualify as a candidate for his more “expanded” test. So instead, he focuses on one stylometry test for one purpose, and another for another purpose. He ends up with two Black Boxes that suit two different purposes—but neither one is legitimate.]

To further illustrate the fallacy of this method, consider the case of two identical twins. Many people have trouble telling them apart, since there are hundreds of similarities in their physical characteristics, and even in their personalities and behaviors. However, it takes only one feature to tell them apart — perhaps one’s nose is a little different than the other’s nose. Their myriad similarities do not make them the same person. We see, then, that it is necessary to focus on distinguishing characteristics rather than on similarities, or we risk being fooled.

Neville alters the phrase great joy and then claims that since Winchester used the word joy a number of times, his writing is similar to the writing in the Central America editorials. Does such a similarity really identify him as the author of the editorials? How many other people use the word joy? Millions! Did you use it recently? If so, by Neville’s way of thinking, maybe you wrote the Central America editorials. Nor are any of Neville’s other cherry-picked similarities informative about the authorship of the editorials.

[Brother Roper’s twin example contradicts his point. First he claims to have found Joseph Smith to be identical to the author of the 900 words—but then he establishes criteria for the “expanded analysis” that excludes Joseph Smith as even a candidate. It’s as if the identical twins are indistinguishable—except one is a zebra and the other is a dolphin.

Of course “joy” is not a highly distinguishing term, so Brother Roper would focus on that. But what about Zarahemla? How many of the potential authors used that term within the year?]

We put Neville’s “similarity words” to the test. The statistical technique of stepwise discriminant analysis examines the groups within a data set to determine the features within the data that are the most distinguishing (discriminating) among the groups. It picks the most distinguishing feature first and subsequent features in descending order of distinctiveness. Applying stepwise discriminant analysis to the expanded set [Page 58]of candidate authors, we found that only 14 of Neville’s words were even slightly distinguishing features among the authors. Therefore he is correct that his words show similarity, but his word list also shows that all the authors use those words similarly. This is depicted in Figure 15.

Since Winchester’s range of word-use frequencies for Neville’s similarity words spans the range for those words in the Central American editorials, he is “similar,” but all the authors’ ranges overlap with the range in the editorials completely or mostly. So if Neville wants to conclude that Winchester wrote the Central America editorials based on his “similarity words,” he must also conclude that at least six of the other authors did so as well, and maybe even the other two.

[While I disagree with what Brother Roper is saying here, I’ll take it as a concession that Winchester is a viable candidate. Now, let’s look at means, motive and opportunity. Anyone who does so will reach the same conclusions that I have, although Phelps and Wm. Smith are developed more in my Second Edition.]

Using Neville’s “similarities” approach, we could pick any one of the nine authors and claim he was the author of the unsigned editorials.

[I’m sure Brother Roper, unconstrained by historical context and facts as he is, could pick any of these. But of course Brother Roper doesn’t mention that Joseph Smith doesn’t even enter this race.]

Winchester is not a materially better choice than any of the others. Incidentally, William Smith, whom Neville also suggests as the author of the Central America editorials, is the least likely choice, since he has least overlap of Neville’s similarities.

Figure 15: Ranges of Word-Use Frequencies for Neville’s “Similarities” for the Expanded Set of Comparison Authors and for the Central America Editorials. Winchester’s range of word-use frequencies is “similar” to that in the Central America editorials, since his range spans that of the editorials, but that is true for six of the other authors as well.

At best, similarities can only generate questions. In the case of Neville’s book, it would be “Could Winchester be the author of the Central America editorials?” Neville cannot validly [Page 59]assert that the supposed similarities he claims to have found answer this question.

To quote Neville’s own words, “ Until now, these facts were never put together. The various threads … woven here were loose strands, unattached, unimportant, and unnoticed. Until now, they’ve been meaningless” (p. 118). Despite Neville’s claims, they are still meaningless.

Other Specious Arguments

• Neville implies that if an article is unsigned, the author “desired anonymity,” as if the real author had something to hide (p. 218).

• He suggests that someone wanting anonymity could alter their style and include phrases borrowed from Joseph Smith to imitate the Prophet (p. 218).

[Actually, this is a well-established practice that bears further research. See this article, for example.]

• To support his case, Neville cites an example of William Smith borrowing wording from Don Carlos Smith (pp. 218, 229).

• Among his similarity arguments he asserts that “the proximity of these uses over less than a year suggests a connection between Winchester and these editorials” (p. 228).

>> Hundreds of unsigned articles were published in this time period in myriads of periodicals. It is unjustifiable to conclude that all anonymous authors were trying to hide something.

[No one is claiming anything of the sort.]

Rather, this was just simply part of common editorial practice in those days, as it is today. Should we conclude that all the unsigned editorials in Winchester’s own newspaper, Gospel Reflector, were not signed because Winchester was trying not to reveal his identity in his own publication and thus had something to hide? Why apply such a claim to unsigned Times and Seasons editorials?

[The difference is evident in the 15 Sep and 1 Oct issues themselves. They both contained significant letters written by Joseph Smith and sent to the editor for publication. His authorship was highlighted and promoted. The contrast with the unsigned editorials couldn’t be more stark.]

Winchester himself, in the Gospel Reflector, borrows many phrases from others, as Roper has shown in the first article of this series.

[Yes, but Brother Roper made some historical errors there, too.]

This is one of the reasons contextual words are not reliable as distinguishing markers of authorship. Stylometric researcher John Hilton stated, “Our wordprinting technique has shown that most highly skilled authors (e.g., Twain, [Page 60]Johnson, Heinlein, etc.), when intentionally trying to imitate the writings of different persons, are unable to successfully change their own free-flow non-contextual word patterns enough to simulate a different wordprint.”⁵⁷

[Contra, read this. And highly skilled authors don’t typically “borrow many phrases from others,” as Winchester and the other early writers did. More important, I contend that these 900 words were edited, which confuses the situation even more in terms of stylometry.]

The possibility that William Smith may have borrowed phrases is irrelevant to the question of Winchester‘s authorship of the Central America editorials. There is nothing informative about one author employing in his writing useful phrases he or she may have found elsewhere.

[Seriously? So if an author borrows phrases from elsewhere, that has no bearing on identifying the author? I can’t wait to read how stylometry addresses that one. Especially when the writing sample is less than 300 words and consists largely of borrowed phrases.]

Further, Neville’s “proximity argument” is a spurious assertion. It is merely “guilt by association.” It is prima fascia [facie] obvious that many things can be in proximity and not be connected. Neville relies only on circumstantial evidence.

Neville’s Basic Conclusions

• He says, “In my view, the results of both analyses [by Roper and by Lund] contradict the conclusions of their authors” (p. 218).

• He says, “Evidence suggests Winchester wrote these” (p. 227).

>> Neville’s view is based only on his preconceived bias against the results of others’ research and his propensity to replace facts with his imagination.

[This isn’t even clever rhetoric. First, it’s counterfactual; I could have had no preconceived bias because I had never heard of Winchester before I started investigating. It’s true that I thought Brother Roper’s data contradicted his conclusion, but that’s because of his own graphics—interpreting them visually the way he did in his first paper. I have replaced no facts; to the contrary, I have incorporated every fact I could find. By contrast, I’ve shown that all seven of the “facts” listed in Brother Roper’s initial article were wrong or misleading. Instead of relying on facts, Brother Roper’s claims fall in the “missing data” category; i.e., everything Brother Roper claims Joseph Smith did and wrote was unobserved, unnoticed, and unrecorded. Everything Joseph did and said that was observed, noticed, and recorded contradicts Brother Roper’s position.]

Neville’s stylometric assertions are ill-informed and baseless. None of Neville’s pseudo-stylometric statements are supported by the evidence from appropriate analyses. He finds confirmatory evidence because all he is looking for is confirmation of his theory. He seems to be in love with his theory; and, like a love-struck suitor, everything he sees confirms his ardor. The dispassionate, skeptical eyes of a historian, statistician and stylometrician look at numerous objective statistical tests and see no persuasive evidence that Winchester authored the unsigned Central America editorials.

[Hmm, will the dispassionate, skeptical historian, statistician and stylometrician please stand up? That description doesn’t fit the authors of this so-called study.

Any readers who have made it this far should know that I commit to a full, robust, and open stylometry analysis of Brother Roper’s data—assuming he ever makes it available to me.

I have no dog in this race, apart from getting at the truth. That’s how I started out and that’s why I approached Brother Roper in January 2014. So far, as evident in this article, he has declined to collaborate in the pursuit of truth. He has declined to open the Black Box for all to see what’s inside.

I think the reason is evident by the hash he’s made of the stylometry.]