[I have
mixed feelings about even responding to Brother Roper’s stylometry article, found below, which was published in the Interpreter. My every
effort in these fields (Church history and Book of Mormon geography) is to seek
consensus about the facts and the most rational conclusions based on those
facts, all with the goal of building the faith of those who believe in the
historicity of the Book of Mormon and the credibility of its first witnesses.
So on one
hand, I’ve been inclined to ignore this article, partly because of its tone but
also because of its misleading content. I have difficulty believing anyone
takes this study seriously, apart from those whose interest consists of
confirming their biases, such as the editors of the Interpreter and other staunch Mesoamerican proponents. The overall
complaint in the article is humorous. Last January, I specifically requested the underlying data and sought to collaborate on a full and robust stylometry analysis; after initially agreeing, Brother Roper refused to proceed. Consequently, I was left with only
what was reported in the original Maxwell Institute paper, found here. Now they criticize me for not having
analyzed the data they refused to provide me!
On the
other hand, there may be some interested readers who could be misled by the
article if no one points out the fallacies. And we can be sure the Interpreter will never publish that.
Hence, this
response.
My overall
conclusion: this last-gasp effort to preserve the Mesoamerican geography is
even worse than I expected. As before, the authors provide no information about
their data, their software, or their parameters; their results cannot be
replicated. In fact, they provide even less data about their new analysis than
they did about their original one.
The article
makes a hash of what was, in their original article, at least coherent. Anyone
who reads to the bitter end will see that the criteria they apply to
Winchester, Phelps, Wm Smith and others flatly contradict the criteria they
apply to Joseph Smith, Woodruff, and Taylor. The authors utterly fail to
address the many historical errors they committed in their first article, but
add more here.
Bottom line: people who want to believe
Joseph Smith wrote the ridiculous, factually false articles in the Times and Seasons can use this article
to confirm their bias, but no objective reader will come away persuaded by the stylometry arguments.
Note:
Throughout this article I refer to Brother Roper. This is not a slight to
Brother Fields or Brother Bassist, but Brother Roper is listed first and for
simplicity I can only refer to one author. I’ll assume that each of the three contributed equally to every part of the article. Perhaps a stylometry
analysis would prove otherwise…except I don’t think stylometry is any good for
co-authored and edited material.
J]
Abstract: This article is the third in a series of three articles
responding to the recent assertion by Jonathan Neville that Benjamin Winchester
was the anonymous author of three unsigned editorials published in Nauvoo in
1842 in the Times and Seasons. The topic of the unsigned
editorials was the possible relationship of archeological discoveries in
Central America to places described in the Book of Mormon narrative. The first
article shows that, contrary to Neville’s claims, Winchester was not a
proponent of a Mesoamerican setting for the Book of Mormon, but rather a
hemispheric one.
[I’ve shown that Brother Roper’s
assertions in that review were specious, but I was grateful to Brother Roper for
helping to make my argument. In fact, I incorporated some of his observations
into my second edition because they corroborate what I wrote in the first
edition.]
Since
this was a view commonly held by early Mormons, his ideas did not warrant any
anonymity for their dissemination.
[The 15 Sep and 1 Oct T&S
articles are unique; no one before or after put Zarahemla in Quirigua or
claimed “we read” in the Book of Mormon that Lehi landed in Central America.
The “commonly held” view was expressed by Orson Pratt; i.e., that Zarahemla was
in South America. Although Joseph explicitly rejected that view, it resurfaced
in 1879 when Pratt added footnotes to the Book of Mormon, which John Taylor
presumably approved.]
The
second article shows that, also contrary to Neville’s claims, Joseph Smith was
not opposed to considering Central American geographic parallels to the Book of
Mormon. The Prophet even seemed to find such possibilities interesting and
supportive of the Book of Mormon.
[There is zero direct evidence
to support this assertion, and Brother Roper’s assertions are based on a series
of unsubstantiated assumptions that contradict the historical evidence.]
This
third article shows that despite Neville’s circumstantial speculations, the
historical and stylometric evidence is overwhelmingly against Winchester as the
author of the Central America editorials.
[The historical evidence shows
Winchester, Phelps, and Wm. Smith were the main contributors to the Times and
Seasons after May 1842, through October 1, 1842, and no legitimate stylometric evidence
contradicts this.]
In The Lost City of Zarahemla from Iowa to Guatemala — and Back Again,
novelist Jonathan Neville tries to discredit what he calls the “limited
Mesoamerican geography” of Book of Mormon events.1 To do so he argues that
Benjamin Winchester — an early Mormon missionary, writer and eventual apostate
— was the anonymous author of three unsigned editorials published in the Times and Seasons on correspondences between the
discoveries in Central America by Stephens [Page 14]and
Catherwood and the Book of Mormon.2 3 The three editorials were
published during Joseph Smith’s editorship of the Times and Seasons from March to November of 1842.
Neville claims that Joseph Smith was opposed to drawing
Mesoamerican connections with the Book of Mormon account and felt that they
posed a threat to his prophetic authority. In order to explain how the three
editorials came to be published, Neville invents an elaborate tale of
subterfuge and conspiracy masterminded by Winchester.
[It’s strange that Brother Roper calls me a novelist when everything
I wrote in the Zarahemla book is
fully documented, while Brother Roper is forced to invent all kinds of
activities Joseph supposedly engaged in that are nowhere supported in the
historical record.]
Neville also includes a pseudo-stylometric “analysis” in an
attempt to support his speculations. Neville is the author of at least twelve
self-published novels. He is an attorney by training. He is not a historian,
statistician, or stylometrician.
[I readily admit I’m not a professional historian, statistician, or
stylometrician, but neither is Brother Roper. A good sign of a losing argument
is over-reliance on “expert testimony” when the expertise of the experts has
not been established and their work is so easily impeached. That said, I
welcome an open, professional analysis if anyone wants to try one.
As I point out in the book, I focused on the historical context and the
content of the articles. I explained in the book that I’m dubious of stylometry
generally, and especially “black-box” stylometry that doesn’t disclose the
underlying data, the software used, the specific assumptions made, or the
specific parameters applied.
The limited stylometry work previously published by these authors has
been criticized by others, but mostly it is ignored. As it should be.
That this paper and the other two reviews were published in the Interpreter means they were not
peer-reviewed and are intended to advance a particular viewpoint. Those who
seek merely to confirm their beliefs that Joseph Smith didn't know much about the Book of Mormon, speculated about its geography, and wrote anonymous articles need not read my response. The Interpreter surely won’t publish it. People
ultimately choose what they want to believe, often in spite of the evidence. If that were
not the case, the Mesoamerican theory never would have gained the prominence it
once had, and its demise would have occurred long ago. The positions expressed in Brother Roper’s article, in my view,
contradict the historical evidence and any logical, rational inferences that
can be drawn therefrom.]
This paper is the third in a series of three articles that
address Neville’s assertions. In the first article Matthew Roper showed that
what Neville characterized as the “limited Mesoamerican geography” of the Book
of Mormon was actually the traditional hemispheric view, which assumed that
North and South America were the lands described in the Book of Mormon and that
Central America was the “narrow neck of land” referred to in the account.4 Winchester’s writings merely
reflected that commonly held perspective, which was never challenged during
Joseph Smith’s lifetime. Neville claims that Winchester and possibly a
co-conspirator had to conceal their identities in order to get their ideas
published. But, since the idea that the “narrow neck of land” was in Central
America was a widely held view, Mormons did not think of it as controversial,
and the motive for a secret publication scheme evaporates. Winchester’s
writings did not present anything especially new or controversial; thus there
was no need for subterfuge and conspiracy to publish them.
[Brother Roper ignores two key facts: 1) people writing in the Times and Seasons were using pseudonyms,
and articles were published anonymously. Why? This point will resurface
shortly. More important here, if these Central American articles were not new or controversial, why has Brother Roper worked so hard to link Joseph Smith to them?]
In the second article Roper discussed the influence of Stephens
and Catherwood’s work Incidents of Travel in Central
America on early [Page 15]Latter-day
Saint readers, including Joseph Smith.5 The Prophet embraced with
interest and enthusiasm the book’s description of Central American history and
ancient ruins, and asserted they corresponded with and supported the Book of
Mormon account.
[The historical record contains zero evidence of Joseph Smith’s personal
interest in Central America; at most, one can draw inferences from anonymous
articles and the Bernhisel note, the author of which is also unknown. One
paragraph earlier, Brother Roper claimed that there was no need for anyone to
write articles in secret. But if there was no need, then why are these articles
anonymous? Why would Joseph Smith write anything anonymously? Of all people, he
is the least likely to spend time
writing something, only to leave his name off. In both of the very issues these
articles were published in—15 Sep and 1 Oct 1842—Joseph had other letters
published that emphasized him as
their author. Ironically, for those who promote a Mesoamerican geography, it is
a good thing that these articles were anonymous, because now they can claim
that Joseph wrote them—in spite of the historical evidence to the contrary.]
In this third article we apply statistical and stylometric
analyses to examine whether Winchester is a likely author of the three unsigned
Central America editorials. We first summarize the results of our previous
paper — “Joseph Smith, The Times and Seasons,
and Central American Ruins” — regarding the Central America editorials, since
Neville used that article to form the foundational premise for his book.6 We show that his premise is
invalid, and therefore the entire argument put forth in his book is baseless.
However, going further to address the specific assertions in his book, we
explain “stylometry,” discuss Neville’s pseudo-stylometry, and present the
results of appropriate stylometric analyses.7 Our results show consistently
that Winchester is not a viable candidate author of the Central America
editorials, and there is no evidence that he is a better candidate than Joseph
Smith.
Summary of “Joseph Smith, the Times and Seasons, and
Central American Ruins”
Candidate
Authors Used: Joseph Smith, John Taylor and Wilford Woodruff
As we discussed in “Joseph Smith, the Times and Seasons, and Central American Ruins,” there
are sound reasons to consider Joseph Smith, John Taylor, and Wilford Woodruff
as potential candidate authors of the unsigned material on Central America and
the Book of Mormon. All three men were in Nauvoo during the time of
publication, they were responsible for the publication of the Times and Seasons, and they were all
familiar with Stephens and Catherwood’s work. As Roper shows in the second
article of this series, Joseph Smith’s letter to John Bernhisel shows that
Joseph Smith shared the enthusiasm of his companions about [Page
16]the correspondences between Central American history, Stephens
and Catherwood’s discoveries, and the Book of Mormon.
[Most of the precedent paragraph is fantasy, not fact. Brother Roper’s
initial article falsely claimed that John Taylor wrote the Bernhisel letter.
There is no evidence that anyone other than Woodruff knew or cared about
Stephens, and Woodruff was ill and absent in the months leading up to the 15
Sept publication of the first two editorials. Worse, Brother Roper himself
already excluded Woodruff as a candidate. A potential author's
physical presence in Nauvoo is irrelevant; most of the material that appears in
the Times and Seasons was mailed in.
And not only is there no evidence that Taylor and Woodruff were working at the Times and Seasons prior to the 15 September issue, there
is strong evidence that they were not
doing so.]
On February 19, 1842, after Joseph Smith had taken control of
the Times and Seasons, Wilford Woodruff wrote in his
journal, “Joseph the Seer is now the Editor of that paper & Elder Taylor
assists him in writing while it has fallen to my lot to take charge of the
Business part of the establishment.”8
[Note that Woodruff’s statement was in February. At that point, I
agree Joseph was working as editor and writer; after all, he was publishing the
Book of Abraham. His journal discusses his activities at the paper in this time
frame. But his activity at the paper soon declined to nothing, both in reality
and as reflected in his journal.]
In a later recollection, John Taylor provided some insight into
what it was like writing for the Prophet and then having him critique and
correct what John Taylor had written. The subject on one occasion had to do
with priesthood keys, the judgment, and the Ancient of Days.
In speaking with the Prophet Joseph once on this subject, he
traced it from the first down to the last, until he got to the Ancient of
Days. He wished me to write something for him on this subject, but I
found it a very difficult thing to do. He had to correct me several
times. We are told that the “judgment shall sit and the books be
opened.” He spoke of the various dispensations and of those holding the keys
thereof, and said there would then be a general giving up or accounting
for. I wrote that each one holding the keys of the several
dispensations would deliver them up to his predecessor, from one to another,
until the whole kingdom should be delivered up to the Father, and then God
would be “all in all.” Said he, “That is not right.” I wrote it
again, and again he said it was not right. It is very difficult to
find language suitable to convey the meaning of spiritual things. The idea was
that they should deliver up or give an account of their administrations, in
their several dispensations, but that they would all retain their several
positions and Priesthood. The Bible and Doctrine and Covenants speak about
certain books which should be opened; and another book would be opened, called
the Book of Life, and out of the things written in these books would men be
judged at the last day.9
[John Taylor said this in 1876 at a funeral service in Salt Lake
City. Nothing in this passage suggest or implies Taylor was referring to the
1842 Times and Seasons while Joseph
was the nominal editor. John Taylor knew and worked with Joseph both before and
after that time frame. The subject matter of the editorials—placing the Book of
Mormon in Mesoamerica—is hardly the sort of “spiritual things” for which it is
“very difficult to find language suitable to convey.” Joseph Smith’s writings,
including the Doctrine and Covenants, have a few things to say about
Priesthood, keys, and judgment. By contrast, not a single one of his writings
mentions Central America or Stephens.]
John Taylor’s account suggests that in working with John Taylor,
Joseph Smith would sometimes explain in his own language what he wanted
written. Then John Taylor would write, after which the Prophet [Page
17]would critique and correct, sometimes repeatedly, what John
Taylor wrote if needed. While we do not know if the same process was followed
in all the writing done under his direction, it does suggest that Joseph Smith
could be very involved in the process, particularly if he considered it a
matter of significant doctrinal importance.
[This is a straw man argument. No one suggests Joseph wouldn’t
critique others in their writing. That’s a far cry from claiming Joseph wrote
(or dictated) unsigned articles in the Times
and Seasons.]
The Prophet also placed Incidents of Travel in Central
America, Chiapas, and Yucatan in the Nauvoo Library and
Literary Institute — a strange decision if he disapproved of the use of the
books by John Taylor, Wilford Woodruff, and others.10
[Brother Roper’s argument here is important to assess because it is
typical of his approach in his other work. Here, in one sentence, Brother Roper
offers a trifecta of poorly reasoned arguments.
First, Brother Roper adopts the Michael D. Quinn argument from Early Mormonism and the Magic World View;
i.e., Brother Roper, like Quinn, claims that Joseph read every book he owned or
had access too. I disagree with both Quinn and Roper on this point. There is
little historical evidence Joseph read much other than the scriptures. In
1841-2, the only book his journal shows him reading is the Book of Mormon.
Second, Brother Roper leaves out key facts. Whenever you read
something Brother Roper has written, it’s a good idea to check his sources (as Earl Wunderli pointed on with respect to Brother Roper's review of his book). This is one of the reasons
I don’t think the Interpreter peer
reviewed this piece, except by friends who are pre-disposed to endorse his beliefs regardless—and often in spite of—the evidence. Brother Roper
finds just enough evidence to support his preconceptions and then he stops.
Review of Edwards on the Will
Life of Tecumseh
Whepleys Compend
Scotts Poetical Works in 5 vols
Gillmores Lectures
Merrills Harmony
Epicureo
Krumanachers Works
Catholic Piety
Home Physician
Apochryphal Testament
Bruns’ Travels
Rebel & other Travels
Browns’ Appeal. gram
Browns English Syntascope
Studies in Poetry & Prose
Old World & the New – Vol 1st
Voyage & Travels of Ross Perry &
others
Bennetts Book Keeping 2 Copies
Incidents of Travel in Yucatan by Stephens 2
Vo
Stephens Travels in Central America 2 Vo
Mosheims Church History 1 Vol 1.50
Times & Seasons 1 2 & 3 Vol also Vol
1 & 2
Dick’s Philosophy
Millenium & other Poems
Beaumonts Experiments
Dictionary of the Holy Bible
Parkers Lectures on Universalism
Sanders Discourse
Metropolitan
Goodrich’s History of the United States
Doddridges Sermons
Catholic Manuel
Whelpleys Compend
Herveys Meditations
Historia de Charles
Rollin 2 Vol
Book of Mormon
By Brother Roper’s logic, Joseph was a well-read librarian--a speed reader, even. By contrast, we do have a
detailed record of his reading habits in this time frame. Wilford Woodruff’s
journal notes that Joseph started reading the Book of Mormon on 5 December
1841. On 15 January, 1842, Joseph’s journal notes he had reached page 54. That
is 54 pages over 41 days, or about 1.3 pages/day. At his rate, it would have
taken Joseph nearly two years to read the Stephens books (not to mention all the other books he donated to the Nauvoo library). Brother Roper ignores this actual data in favor of his unsupported assumption
that Joseph read the Stephens books in six weeks or less (based on the time
between Woodruff's arrival in Nauvoo and the date of the Bernhisel note). And
all of this took place without anyone else commenting on Joseph’s sudden
interest in Central America or his new—and temporary—speedreading skill.
Furthermore, Brother Roper’s argument
requires that Joseph must have not only read all these books, but approved “the
use of the books by John Taylor, Wilford Woodruff, and others.” I’d be
interested in whatever evidence Roper can produce of Taylor, Woodruff, or
anyone else using the Catholic Manuel
or Catholic Piety, or of Joseph Smith approving such use.
Bottom line, Joseph's donation of the Stephens books puts them in
the same category as Catholic Piety. The donation has zero
evidentiary value regarding his supposed enthusiasm for them as proof that the
Book of Mormon cities were in Central America.
Now for
the third faulty argument in the trifecta.
Brother Roper assumes his
theory is fact (i.e., that Taylor, Woodruff and others “used” the Stephens
book, when the only evidence is that Woodruff read them and an anonymous person used them in unsigned editorials). Then he uses his unsupported
theory to form a false argument by turning my argument inside out. Nowhere have
I written or suggested that Joseph Smith disapproved of these books. In my
view, there is no evidence he cared about them at all, any more than he cared
about Catholic Piety. In addition, it
is not inconsistent with what Joseph actually said and wrote to consider
post-Book of Mormon ruins in Central America as evidence of post-Book of Mormon
people who may have included migrants from Lamanite territory. In fact, the
Aztalan article in the Times and Seasons
makes that very argument. Thus, there is no problem with people citing Stephens
as general evidence of the Book of Mormon; the problem is claiming the text
says Lehi landed there and that Zarahemla is in Quirigua (or nearby), which are
the unique features of the anonymous editorials published on Sept. 15 and Oct.
1.]
John Taylor and Wilford Woodruff were seriously ill during
August and September, and both men were essentially bedridden from August 9
until middle to late September. This suggests the chance that someone besides
them might have penned the editorials for September 15 and October 1. Neville
thinks that Joseph Smith could not have contributed to the content of any of
the articles because he was in hiding from his enemies in September 1842 and
may not have been able to visit the printing office during that time.
[This is not my argument. Brother Roper borrowed this from somewhere
else—another common characteristic of his three reviews, as I’ve noted. My
argument is based on simple facts. Joseph wrote a letter on 1 Sept and sent it
to the editor of the Times and Seasons,
who published it on 15 Sept. He wrote another one on 6 Sept and sent it along.
This one was published on 1 Oct. In both cases, the editor publicized Joseph’s
authorship of the articles, knowing that such communications were highly valued
by his readers. Based on these facts, it is certainly possible that Joseph could
have written the 900 words and provided the several thousand words of extracts.
I’ve never said otherwise.
It’s not a question of whether he could have, but whether he did.
Brother Roper takes the position that while 1) Joseph signed the two letters
that became Sections 127 and 128 in the Doctrine and Covenants and 2) sent them
to the editor for publication, and 3) the editor introduced them in the
newspaper by touting Joseph’s authorship, for some reason Joseph separately 1)
wrote the three short comments, 2) attached them to extensive extracts from
Stephens, 3) published them himself (or directed the editor to publish them) 4)
anonymously. There could not be a more stark disparity between the treatment
and content of the two sets of documents.]
None of the editorial portions of the Central America articles
for 1842 were long, and they would not have required an inordinate amount of
time to write. [This is a remarkable argument. What is
“inordinate” under the circumstances? Readers should think about this. The 900 words are not long, granted. But the
extracts are carefully selected from the 900 pages of the Stephens books. Joseph supposedly read these books
nearly a year earlier. Now, in September, when he’s evading the authorities and
seeking assistance from the government and friends, when he’s focusing on the
temple to write Sections 127 and 128, when he’s dictating the Book of the Law
of the Lord, handling the real estate transactions in Nauvoo, managing the
bankruptcy he declared a few months earlier, and so on, he supposedly took the
time to find important extracts from the 900 pages of Stephens books and write
editorials about them. Just in terms of logistics, this is unlikely. On top of
that, he had to instruct his scribes not to mention all his work with Stephens,
tell everyone he knew not to mention it, and make sure the editor of the Times and Seasons kept his work
anonymous—exactly the opposite of what he did with sections 127 and 128. In
Brother Roper’s version of events, Joseph’s involvement with the Stephens books
was the best-kept secret in Nauvoo. It was kept more secret than plural
marriage.]
The editorials for July 15 (signed “Ed”) and September 15
(unsigned) both [there are actually two in September] mention Stephens and
Catherwood’s discoveries and have a similar theme. The September 15 material
may have been written in September, but it could have been written just as
easily previous to John Taylor’s and Wilford Woodruff’s illnesses in July or
early August.
[The editorial in July was primarily a 1,200-word extract from
Josiah Priest’s American Antiquities.
Benjamin Winchester is the only contributor to the Times and Seasons during 1842 known to have had a copy of Priest’s
book. There is no evidence that anyone
in Nauvoo had a copy. Brother Roper not only has Joseph writing anonymous
articles, but now he has him extracting books he doesn’t own.
As I’ve pointed out, the articles could have been written by literally
anyone with access to the mail. (Parley P. Pratt quoted from American Antiquities in his Voice of Warning, but there’s no
evidence he owned a copy of his own, and besides, he was in England in 1842. Of
course, he could have mailed in this article like anyone else, but why would he
insist on anonymity?) Theoretically, the articles could even be holdovers from deceased
persons such as Don Carlos. That’s the point: they are anonymous. Brother Roper has concocted a false, counterfactual
requirement that these articles had to have been written by someone living in
Nauvoo.]
Neville notes that in Joseph Smith’s journal, “There is no hint
that Joseph is reading, writing, or conversing about any topic related to Book
of Mormon geography” (p. 130). While true, the way that Neville presents this
information is misleading. Indeed, as far as we are aware, there is no hint of any discussion of Book of Mormon geography in anyone’s Nauvoo journal during 1842.
[??? This is precisely my point! Apart from Wilford Woodruff’s
journal in September 1841 and the Bernhisel note in November 1841, no one in Nauvoo cared about Book of Mormon
geography. Here’s another example. In May 1841, a man from New York
presented Joseph Smith with a 20-foot long scroll that had been taken from a
rock in South America that purported to depict Lehi crossing the sea and other
Book of Mormon events. Few people know about this incident because neither
Joseph nor anyone close to him gave it any credence.
Brother Roper here is supporting my argument. The people living in
Nauvoo were focused on building the temple, building the city, farming,
welcoming new Saints, and sending out missionaries. It was the missionaries in
the field, such as Winchester in Philadelphia, who were dealing on a daily basis with
the missionary effort, battling with the press and antagonistic ministers, etc. That’s another reason why I think it was someone outside Nauvoo mailing the Priest and
Stephens material to the newspaper. And most of the material published in the Times and Seasons was mailed in.]
Thus, the journal’s silence cannot be taken to mean very much,
since someone was interested in it, and the Times and Seasons published a handful of articles
relating to that subject while Joseph Smith was editor, despite the subject’s
absence from anyone’s personal diaries.
[Well. Start with “the journal.” Joseph Smith’s journal
was not like everyone else’s. It was kept by scribes; it was kept daily; it was
kept in response to a commandment; and it was relatively meticulous. Everything
we know he wrote, including the Book of Abraham and the Book of the Law of the
Lord, is accounted for in the 1842 journal. To have such major pronouncements
as we have in the 900 words (let alone the extracts from American Antiquities), with no mention of them in the journal, is exceptional. Unique, even. Next, we have Woodruff’s journal, probably the most complete
apart from Joseph’s, in which he claimed to try to write down everything Joseph
said. What did Woodruff record regarding Joseph’s comments about Stephens?
Nothing. We have Phelps and Woodruff and others writing letters, mentioning
Joseph and his activities, but no mention of Stephens.
Now look at the second component of Brother Roper’s argument.
“Someone” was interested in it, and because Joseph’s journal doesn’t indicate
it was Joseph, Brother Roper concludes it must
be Joseph. He relies on Joseph’s role as editor, for which the evidence is
as threadbare as Brother Roper’s authorship claim. It’s fascinating that of all
the things actually recorded in Joseph’s journal, only the things Brother Roper claims Joseph did are missing.
Brother Roper remains oblivious to how the evidence he cites
supports my argument, not his. The fact that no one in Nauvoo mentioned these
lengthy books, despite their (according to Brother Roper) tremendous
importance, is evidence that it was not anyone living in Nauvoo who wrote the
articles. It is not evidence that people simply forgot to mention Stephens (or
Priest’s American Antiquities) in
their journals.]
Figure 1: Discriminant
Analysis Plot from “Joseph Smith, the Times and Seasons, and
Central American Ruins.”11 The
Joseph Smith holographic texts, editorials signed “Joseph Smith,” editorials
signed “Ed.” during his editorship, and the unsigned editorials during his
editorship cluster together as a group and are obviously separate from the John
Taylor and Wilford Woodruff texts. The editorials on the topic of Central
America cluster closest to the Joseph Smith Group.
[Page
18]Joseph Smith’s journal does not record everything that Joseph
Smith did or did not do during this time, but it does show that Joseph Smith
was in or near Nauvoo while printing activities were going on, which is why he
must be considered seriously as a candidate author.
[Another
straw man argument. No one suggests the journals record everything Joseph did,
but they do record what he deemed
most important. He was in or near Nauvoo when horses were being shoed and meals
were being prepared; does that mean he spent his time shoeing horses and
cooking meals? Besides, it’s not merely the absence of Joseph’s alleged editing
activities from his journal that is relevant. There is also third-party
evidence to support what the journals indicate about Joseph’s activities; i.e.,
that he went to the printing office less than once a month while he was editor.]
Joseph
Smith’s journal does show that he met with John Taylor on September 21 about
the work of the printing office and also on September 23. This would have given
him and John Taylor the opportunity to read or discuss what was to appear in
the October 1 editorial and at least allow Joseph Smith to provide his own
input, had he wished to do so.
[This is a
misleading argument that underscores the utility of Joseph’s journal. What did
Joseph and Taylor meet about? His journal tells us, and it has nothing to do
with the content of the Times and Seasons!
Joseph wanted to start another paper across the river. He told Taylor to have
Woodruff go over and do it. I point out all of this in my book. A key point here
is, what happened to those plans, and why? Read the book and find out, because Brother
Roper won’t tell you. Instead, he helps make my point that Joseph could have provided his own input, had
he wished to—but he didn't. Neither he nor John Taylor were talking about editorials or
content of the Times and Seasons.]
Methods Used and Results: Our previous article
examined the probable authorship of the Times and Seasons editorials
related to Central America. A timeline of significant events, as well as
other [Page 19]historical evidence, indicate that the most likely candidates
are limited to Joseph Smith, John Taylor and Wilford Woodruff.
[This timeline consists of 7 points, and every one of them is false
or misleading, as I’ve shown. I assume this is obvious to anyone familiar with
Church history, but if not, maybe I’ll have to do a post on that, too.]
To characterize the writing style of each potential author, we
assembled a collection of texts known to have been written by each author.
For Joseph Smith we took a sample of texts for which he is the
known author (holographic documents written in his own hand) and combined them
with texts from the Times and Seasons signed
“Joseph Smith,” editorials signed “Ed” when Joseph Smith was editor, and
unsigned editorials when he was the editor. We refer to these texts together as
the “Joseph Smith Group.”
[This is problematic in multiple grounds. First, Brother Roper
doesn’t list the documents, so we have no idea what he actually used. The one
he does list—the Bernhisel letter—has an unknown
author. Second, holographic writings are different in kind, purpose, and
audience than published writings. Third, we know Joseph didn’t even write some
of the material he signed—Woodruff
noted that Joseph hardly had time to sign his name—so how can we legitimately
include as his writings things we don’t know he wrote? Fourth, Joseph’s
holographic writings date to the early 1830s. Such time disparity is a factor
in assessing writing samples, as we will see. These problems are compounded by
Brother Roper’s refusal to share with his readers what actual documents he
included.
It turns out that Joseph’s holographic writings are collected by the
Joseph Smith Papers here.
If you go to that link you will see that there are only six items dated
1841-1843. Here’s the list:
Agreement with Ebenezer and Elender Wiggins, 14 May 1841
Note of Authorization, 24 February 1842
Complaint against William Thomas, 2 August 1842
Letter to Newel K. and Elizabeth Smith Whitney, 18 August 1842
Letter to Lucien Adams, 2 October 1843
Note to William Clayton, 9 December 1842
None of these—actually, none of Joseph’s holographic
writings—include any mention of Central America, Stephens, or the like.]
We applied the statistical technique of discriminant analysis to
identify how the texts group together based on the word-use frequencies in each
text and then determined the probable group membership of the texts of unknown
authorship.12
[If you think this citation discloses any of Brother Roper’s data,
such as what words were used to determine probably group membership, you’ll be
disappointed. This is a citation to a useless article about theory.]
We showed (see Figure 1) that the Joseph Smith Group of texts
cluster together, and they are distinct from the John Taylor texts and the
Wilford Woodruff texts.
[I missed the description of the John Taylor and Wilford Woodruff
texts. Did these also include holographic material? Articles signed by either
of them but known to have been written by someone else? Articles not signed by
either of them?]
The Central America editorials composited together
[In the original article, Brother Roper listed six editorials—three that included extracts from American Antiquities and three that
included extracts from Stephens. But then he writes, “Our interest is the
authorship of the five Times and Seasons
editorials… To investigate probably authorship of the five small editorials
(two signed “Ed.” and three unsigned) in the Times and Seasons that referred to Central America, we put them
into one composite block of text so there would be sufficient data to measure
word frequencies.”
Think about this for a moment.
First, Brother Roper arbitrarily excludes one of the editorials he
identified because it doesn’t “refer” to Central America. (In fact, three of
the editorials focus more on North America—Kentucky, Wisconsin and
Tennessee—than Central America.)
Second, Brother Roper simply assumes that all five (or six) of the
editorials were written by the same person. This is particularly interesting
because three of them included extracts from the book that only Winchester
owned.
Third, Brother Roper created this composite “so there would be
sufficient data to measure word frequencies.” IOW, each individual article, by
itself, is too short to measure word frequencies. Brother Roper doesn’t cite a
theory of stylometry that justifies combining separate samples by unknown
authors to determine the authorship of the composite.
Fourth, Brother Roper doesn’t tell us which portions of the
editorials he used in the composite. This is important because quotation marks
are optional in much of the Times and
Seasons; other authors have confused some of the extracts with the
editorial portions of the pieces.
It’s not clear from the narrative that Brother Roper even
acknowledges these problems; certainly he doesn’t address them.]
are closest to the Joseph Smith Group of texts and obviously
closer to those texts than to the John Taylor or Wilford Woodruff texts.
[This is meaningless “black-box” scholarship. So far, the stylometry
analysis appears set up to confirm Joseph’s authorship, not to determine who
wrote the anonymous editorials. Apparently we are expected to take Brother
Roper’s word for all of this, but if he’s confident in the methodology and
results, why the secrecy?]
Thus we concluded that the writing style in the Central American
editorials is closest to the writing style of Joseph Smith, and consequently
that Joseph Smith is the most likely author of the Central America editorials
of the three historically justifiable candidate authors.
[These three, even if they did contribute to the Times and Seasons in 1842 beyond their
signed contributions, were minor contributors. Apart from Joseph Smith’s
History, the Book of Abraham, and D&C 127 & 128, Joseph wrote very
little for the 1842 (3d edition) paper. Woodruff wrote one signed article, and
John Taylor none. The main contributors to the third edition of the Times and Seasons were Orson Hyde
(letters from his mission to Israel) and Benjamin Winchester (unsigned articles
from the Gospel Reflector). Most material published in the paper came
through the mail, including the pseudonymous articles from Boston that I
attribute, in part, to Winchester.
Even taking Brother Roper’s results on their face, the most that can
be said is a mixture of unsigned editorials is closer to Joseph’s holographic
writings than it is to Taylor’s and Woodruff’s published writings. But in what
way? We have no idea and Brother Roper isn’t going to tell us.]
The “Lost City” of Zarahemla
In The Lost City of Zarahemla, Neville claims that the
Central America editorials do not belong to the Joseph Smith Group and spends
about 200 pages speculating how Winchester could have been the author. He
repeatedly states his speculations as “fact” without scientific substantiation,
and he even resorts to using a weak stylometric analysis for support.
[To the
contrary, I offer a variety of other possible explanations for the historical data.
I simply conclude that Winchester as author is the most plausible. I
specifically questioned the utility of stylometric analysis generally, but I
did apply the methods described by Lund (who at least tells readers what he
does).
Later in
this paper, Brother Roper will accuse me of using too many unspecific words,
such as might and plausible. I could cite that part of his paper to refute his
assertions above, but I trust readers can figure that out pretty easily.]
Neville’s Foundational Premise is Invalid
Neville
asserts that there must be an unrecognized anonymous author for the Central
America editorials by claiming that Figure 1 shows the Central America
editorials collectively to be an “outlier” (pp. 219-20) in relation to the
other texts in the Joseph Smith Group.
[Actually,
I pointed to all three of the Figures Brother Roper provided. In this paper, he
showed only the front view. In his original paper, he also showed the top and
side views, both of which significantly separate the “Smith” group from the
“Editor” group. Readers can decide whether Brother Roper was being informative
by reproducing only the front view here. As noted previously, it’s always a
good idea to check on the facts Brother Roper represents. You can see the other
two Figures on page 96 here and decide for yourself.]
He says
this leads him to believe that there was a different author other than Joseph
Smith, [Page 20]John Taylor or Wilford Woodruff for the unsigned Central America
editorials. Neville conjectures that Benjamin Winchester was that unrecognized
latent author of the unsigned editorials.
What Neville means by “outlier” is more properly referred to as
an “extreme value.”
[No, I did not mean an “extreme value.” I meant an “outlier.”
Brother Roper’s database is a secret, but the brief description he has provided
suggests fundamental problems, as I already listed. The “Central America”
composite is closest to the “Editor” group (although it is still at some
distance in 2 of the 3 Figures). In the Front and Top views, it is by far the
furthest to the left of all the data points. Here’s how Brother Roper’s
original paper characterized the relevance of distance in Figure 7:
A top view of the three
authors’ texts showing the relative position of the Joseph Smith texts and the
editorials with the texts signed “Editor” spread away from the other texts,
indicating that those texts are somewhat distinguishable from the others. The
Central America composite text is closest to the Joseph Smith text and the
“Editor” texts.
In Figure 7, the “Central America” composite is farther from the
“Smith” and “Editor” texts than the “Editor” text is from “Woodruff” or
“Smith.” Brother Roper claims that distance renders those texts “somewhat
distinguishable” from the others, but next he’s going to complain because I
made the same observation about the
Central America composite.]
The term “outlier” refers to an extreme value among a set of
values that is so far from the other data points that it is probably
incongruent with the other members of that set of values. His method of
assessment is purely visual; and, as he correctly says, it is only his
“layman’s opinion” (p. 219).
[In his comment on his own Figure 7, Brother Roper described the
relative position in visual terms.
Apparently that’s okay because his visual method confirmed his bias. My
application of his own approach is
invalid because it contradicts his preferred belief.]
Yet his claim that the Central America editorials are an
“outlier” is the foundational premise of his entire argument. If this claim is
false, his entire argument has no basis and cannot be substantiated.
[This is another logical fallacy. Brother Roper is taking a
common-sense visual observation that he can’t deny and converting it to a
statistical term of art. And don’t forget, he’s also operating on the unsubstantiated
assumption that all of the samples he characterized as written by Joseph were
in fact written by Joseph and relevant to the question.]
Ignoring rigorous statistical analysis
[I was not ignoring Brother
Roper’s own depiction of his data—I was relying
on it. Anyone who reads Brother Roper’s article sees the same thing I do. He
doesn’t provide any data other than his Figures, and refuses to provide me his
underlying data, software, parameters, etc.]
and only visually
examining the plot in Figure 1, Neville concludes that the Central American
texts are “too far” from the Joseph Smith Group to be congruent in style with
the other texts. He does not realize that the distances shown in the plot are
scaled relative to only the texts examined and do not represent any absolute
measure of separation between the texts, as would be the case if the data
points were simple two-dimensional locations of physical items plotted on a map
in a Cartesian coordinate system.
[Brother Roper described the relative position in Figure 7 as
sufficient to indicate that those texts “are somewhat distinguishable from the
others.” I’m saying the even greater distance makes the “Central America” texts
“somewhat distinguishable from the others.”]
Graphical Tests: His “eyeball
method” is a simplistic approximation of applying a Euclidean distance measure
like the distance-between-two-points calculation taught in high school algebra.13 Figure 2 shows a histogram
plot (frequency plot) of the Euclidean distances of each text in the Joseph
Smith Group from the centroid of the group.14 Along with these we have
included the distances to typical points from the John Taylor and Wilford
Woodruff groups to show what true “outliers” would look like on the plot. The
points for John Taylor and Wilford Woodruff are extreme values, but not the
Central America point. In fact, there is another text in the Joseph Smith Group
even farther away than the Central America editorials.
[This analysis seems intent on undermining the validity of the
original study—or at least Brother Roper’s explanation in that study. Of
course, this should surprise no one; Brother Roper still hasn’t told us what
he’s using as samples of Joseph’s writing, and there is reason to believe that
much of what has been attributed to Joseph was actually written by others. That
some of the other samples are even further away than the Central America
editorials does not mean Joseph wrote them; it raises questions about the
authorship of the comparison samples.]
Page 21]
Figure 2: Histogram of
Euclidean Distances of Each Text in the Joseph Smith Group from the Centroid of
the Group. The distances from the Joseph Smith Group centroid to typical points
from the John Taylor group and the Wilford Woodruff group have been added to
show what truly incongruent values look like. The Central America editorials
are not inconsistent as part of the Joseph Smith Group, and there is even
another Joseph Smith Group text more distant than the Central America
editorials.
A data
point exactly at the centroid of the Joseph Smith Group would have a Euclidean
distance of zero (0.00). The first bar on the left in the plot shows five
points with Euclidean distances between 0.00 and 0.50. The distances are
deviations from the centroid and thus positive with no direction indicated. The
next bar shows twelve points with distances between 0.50 and 1.00. An extreme
value is one that is inconsistent with the rest of the data points in a set of
data. For a point to be an extreme value in the plot, it would be to the right
of the other Joseph Smith Group points, as indicated by a gap between it and
the other group points. There is no such gap for the Central America
editorials, which have a distance of 3.83, but there is an obvious gap in the
distance to the typical John Taylor and Wilford Woodruff points, 6.08 and 6.10
respectively, showing what true extreme values look like. The histogram plot
shows that the Central America editorials are within the distribution of the
other points in the Joseph Smith Group and do not exhibit characteristics of an
extreme value.15
[Only Brother
Roper suggests that Woodruff and/or Taylor are the authors, so this is a red
herring argument. So far, Brother Roper has declined to tell us not only what
writing samples but the rest of his methodology. No one disputes the basic
statistical analysis here, but it is impossible to dispute or agree with the
relevance of what Roper has written here.
Here’s
a short discussion of the problem with confirmation bias and data manipulation
in the context of measuring and adjusting temperatures, but it applies here
pretty well. Later, I’ll make the connection more explicit. Basically, it’s not
the statistics that matter; it’s the underlying data that matter.
“[T]here is
overwhelming evidence that confirmation bias doesn’t require anything like
deliberate dishonesty. All it requires is a failure in applying double blind,
placebo controlled reasoning in measurements. Ask any physician or medical
researcher. It is almost impossible for the human mind not to select data in
ways that confirm our biases if we don’t actively defeat it. It is as difficult
as it is for humans to write down a random number sequence that is at all like
an actual random number sequence (go on, try it, you’ll fail). There are a
thousand small ways to make it so. Simply considering ten adjustments, trying
out all of them on small subsets of the data, and consistently rejecting
corrections that produce a change “with the wrong sign” compared to what you
expect is enough. You can justify all six of the corrections you kept, but you
couldn’t really justify not keeping the ones you reject. That will do it. In
fact, if you truly believe that past temperatures are cooler than present ones,
you will only look for hypotheses to test that lead to past cooling and won’t
even try to think of those that might produce past warming (relative to the
present).
“Why was
NCDC even looking at ocean intake temperatures? Because the global temperature
wasn’t doing what it was supposed to do! Why did Cowtan and Way look at arctic
anomalies? Because temperatures there weren’t doing what they were supposed to
be doing!
“One of the
whole points about error analysis is that one expects a priori error from all
sources to be random, not biased. One source of error might not be random, but
another source of error might not be random as well, in the opposite direction.
All it takes to introduce bias is to correct for all of the errors that are
systematic in one direction, and not even notice sources of error that might
work the other way. It is why correcting data before applying statistics to it,
especially data correction by people who expect the data to point to some
conclusion, is a place that angels rightfully fear to tread. Humans are greedy
pattern matching engines, and it only takes one discovery of a four leaf clover
correlated with winning the lottery to overwhelm all of the billions of four
leaf clovers that exist but somehow don’t affect lottery odds in the minds of
many individuals. We see fluffy sheep in the clouds, and Jesus on a burned
piece of toast.
“But they
aren’t really there.”]
[Page 22]A “box plot” is another standard statistical representation of a
set of data.16 The box plot shown in Figure
3 further illustrates that the Central America editorials are not an extreme
value of the Joseph Smith Group.
The “box” contains the middle 50% of the data points. The
vertical line extending down from the box spans the lower 25% of the data,
ending at the minimum value. The vertical line extending upwards from the box
spans the upper 25% of the data, ending at the maximum value.
The threshold distance for an extreme value for the Joseph Smith
Group using typical box plot methodology is 4.41. The Central America
editorials distance of 3.83 is not beyond the threshold. Therefore, here again,
the Central America editorials are not an extreme value within the Joseph Smith
Group of texts.
[Brother Roper has substituted his own argument for mine. He’s
arguing with himself. I never said the articles represented an extreme
statistical value. Brother Roper himself, in his article, noted that the “Central
America” text fell outside the grouping of his Joseph material, which he
interpreted to mean John Taylor helped with the writing or editing. Now he is
arguing with his own initial article, not my point.]
Figure 3: Box Plot of
Euclidean Distances from the Centroid of Joseph Smith Group with the Extreme
Value Threshold Distance. The Central America editorials distance is not an
extreme value in relation to the other texts in the Joseph Smith Group.
Univariate
Tests: There are numerous objective statistical tests for extreme
values in a set of data: Dixon’s Q Test, Grubb’s Test, Iglewicz and Hoaglin
Test, Rosner’s Generalized Extreme Studentized Deviate [Page
23](GESD) Test, and Tietjen-Moore Test.17 To further test Neville’s
eyeball method, we applied each of these statistical tests to the two-dimensional
distance data in Figure 1. Table 1 shows the results.
[All of
this ignores the point that we still don’t know 1) Brother Roper’s writing
samples or 2) his methodology. He’s running statistical analysis without first
defending—or even explaining—the underlying data. This entire exercise is
pointless. If anything, it illustrates the fundamental weakness of Brother Roper’s
argument: he has no historical evidence to support his assertion, so he’s
misleading readers by using statistical analysis purely on the assumption that his underlying data is
valid.]
Extreme Value Test
|
Central America’s
Test Value
|
Extreme Value
Criteria
|
Result
|
Dixon’s Q
|
0.115
|
>0.206
|
Not Extreme
|
Grubb’s
|
2.379
|
>3.230
|
Not Extreme
|
Iglewicz Y Hoaglin
|
2.343
|
>3.500
|
Not Extreme
|
Rosner’s GESD
|
2.168
|
>2.768
|
Not Extreme
|
Tietjen-Moore
|
0.841
|
<0.751
|
Not Extreme
|
Table
1: Results of Extreme Value Tests of the Euclidean Distances in
Figure 1. All the tests show no evidence that the Central America editorials
are an extreme value within the Joseph Smith Group.
Each test uses the data and calculates a test value for the
Central America editorials. The test value is compared to a criterion value
that indicates whether the Central America editorials might be an extreme value
or not. For example, Dixon’s Q test calculates a test value for the Central
America editorials of 0.116. This is compared to the criterion value of 0.206.
Since 0.116 is not greater than 0.206, there is no evidence that the Central
America editorials are an extreme value. For the first four tests in Table 1,
if the calculated test value is greater than the criterion value, then that
would indicate an extreme value. The Tietjen-Moore test is different. If its
test value is less than the criterion value then this would indicate an extreme
value.
As we can easily see in Table 1, all the tests show no evidence
that the Central America editorials are an extreme value from the other texts
in the Joseph Smith Group.
Multivariate Test: Since the data
measure the proportions each author used 67 noncontextual words, the data
constitute a 67-dimensional multivariate data set. To visualize the
multi-dimensional data in graphical form on paper we needed to depict it in our
previous article in only two dimensions. The amateur Neville sees two-dimensional
plots and thinks this is all the information. Consequently, Neville was
easily [Page 24]deceived by his “eyeball test.”
[This is what I observed at the outset was funny. I actually
requested the underlying data; Brother Roper refused to share it with me.
Consequently, I was left with only what was reported in the original paper. Now
they criticize me for not having analyzed the data they refused to provide me!
In that article, Brother Roper gives his readers two-dimensional depictions of
three-dimensional data, with none of the underlying data. If I was “deceived”
it was by Brother Roper’s own interpretation, in that article, of his own data.
Worse, so far in this article, he has provided us with only one two-dimensional
depiction.]
The most commonly used test for extreme values within a
high-dimensional multivariate data set is to test the Mahalanobis distance.18 For a Mahalanobis distance to
be considered an extreme value, the random sampling chance of observing that
distance is generally required to be less than one in a thousand (probability
< 0.001).19 Using the data from our
previous article, this test shows that the Central America editorials are not
extreme values in comparison to all the other texts in the Joseph Smith Group.
The results are shown in Table 2.
|
Joseph Smith
|
John Taylor
|
Wilford Woodruff
|
Critical Value
|
Mahalnobis Distance
|
12.79
|
72.80
|
100.74
|
13.82
|
Group Membership Probability
|
100.0%
|
0.0%
|
0.0%
|
>95%
|
Table
2. Mahalanobis Distances and Probability of Group Membership
for the Central America Editorials. The Mahalanobis distance from the
editorials to the centroid of the Joseph Smith Group is not beyond the critical
value. The probability of the editorials’ membership in the Joseph Smith Group
is virtually 100%.
Since the Mahalanobis distance from the Central America
editorials to the centroid of the Joseph Smith Group (12.79) is not larger than
the critical value (13.82), while the distances to the centroids of the John
Taylor and Wilford Woodruff groups are larger than the critical value, the
editorials are judged to be “outliers” from John Taylor and Wilford Woodruff
but not from Joseph Smith. The probability of group membership of the
editorials in the Joseph Smith Group is virtually 100%. This further indicates
that there is no evidence the Central America editorials are “outliers” in the
Joseph Smith Group when tested with the appropriate statistical technique.
[Here again, Brother Roper is assessing his own argument, not mine.]
Open-Set Test: Is There Evidence of a Latent Candidate Author? A
statistical technique that can be applied directly to Neville’s claim of
someone other than Joseph Smith, John Taylor or Wilford Woodruff[Page
25]authoring the three unsigned Central America editorials is the
Extended Nearest Shrunken Centroid Method (ENSCM). ENSCM is an extension of a
sophisticated technique developed for high-dimensional classification problems
in genomics research and DNA microarray analysis that we and others have used
in authorship attribution, including our studies of the Book of Mormon.20 21 22 23 24
[These studies are hardly uncontroversial and unchallenged, either.
Mostly they’re ignored. No offense, but I think this article is another reason
why Brother Roper’s analyses should be ignored.]
Using ENSCM, we tested Joseph Smith, John Taylor and Wilford
Woodruff as an open-set of candidate authors of the Central American
editorials. ENSCM first establishes a profile of word-use frequencies for each
candidate author based on texts he or she is known to have written.
[Maybe I missed the detailed list of the texts tested here and the
profile of word-use frequencies. Surely Brother Roper wouldn’t still be keeping
that from us, would he?]
Then ENSCM computes the probability that each candidate author’s
writing style is closest in style to the style of the texts of unknown
authorship. However, ENSCM allows for the possibility of an additional unknown
latent author — sometimes referred to as the “none of the above” alternative.
Should the latent author’s probability of closest writing style exceed the
probability of any of the candidate authors, then the group of authors should
be considered to be an open-set and include the possibility of an unknown
author.
Applying ENSCM, we found that the latent author probability —
the probability someone else needs to be considered as having a writing style
closer than at least one of the candidate authors — is less than one in a
thousand (probability < 0.001). See Figure 4. This means that the word-use
frequencies of the candidate authors are close enough to the word-use
frequencies in the Central America editorials that we can conclude there is
insufficient evidence of the need to consider other authors. Consistent with
the historical evidence, Joseph Smith, John Taylor and [Page
26]Wilford Woodruff can be considered to be a closed-set of
candidates for authorship of the three Central America editorials.
Consequently, in our previous paper we included only Joseph
Smith (and the editorials that group with Joseph Smith), John Taylor, and
Wilford Woodruff in the analysis. Since the question of authorship of the
Central American editorials could be addressed as a closed-set, the evidence
indicates that Joseph Smith is the most probable author, as we concluded in the
previous paper.
Figure 4: Extended Nearest
Shrunken Centroid Method (ENSCM) Probability of Closest Pattern. The
probability of a latent author is less than 0.001. This indicates that Joseph
Smith, John Taylor and Wilford Woodruff (along with “Ed.”) can be considered as
a closed-set of candidates for attributing authorship of the Central America
editorials.
Thus,
Neville’s biased “layman’s opinion” based on his “eyeball test” is contradicted
by a host of relevant objective statistical tests and analyses. Therefore, his
assertion that another author needs to be considered is baseless. Consequently,
the entire foundational premise of Neville’s book is invalid.
[Brother Roper’s
statistical analysis was not the foundational premise of the book; the question
of who wrote the articles was. Brother Roper’s article was so full of
historical errors that I questioned its validity on that basis alone. On top of
that, his portrayal of the 3-dimensional data appeared to contradict his own
conclusion—and it still does. But to the merits of this ENSCM analysis, it’s a
classic case of garbage in, garbage out. Until Brother Roper fully discloses
his data, software, assumptions, parameters and methodology, he is reporting on
nothing more than a Black Box outcome. That’s ideal for confirmation bias, but
entirely unpersuasive for anything else.]
Adding Benjamin Winchester to the Analysis
We have
shown that the foundational premise of Neville’s argument is invalid;
nevertheless, to directly test Neville’s contention, we added Winchester to the
[still
undisclosed] data from our 2013 article and reanalyzed the data to see how
close his style is to the style in the [arbitrarily composited] Central
America editorials. We also tested for evidence that his style is closer to the
editorials than to Joseph Smith’s. We followed objective, formal [and still
secret] scientific hypothesis-testing methodology.
We formulate the research question as follows:
[Page 27]Is Benjamin Winchester’s writing style the same as the writing
style in the Central America editorials, and is his style closer to that of the
editorials than to Joseph Smith’s style?
To answer this research question we formulate the null (H0) and
alternative (Ha) hypotheses as follows:
H0: The Central America editorials writing style is closest to Joseph
Smith’s style. [using still secret writing samples]
Ha: The Central America editorials writing style is closest to
Benjamin Winchester’s style. [using still secret writing
samples]
We performed [still secret] discriminant analysis and
determined the probabilities of [still secret] group
membership. Figure 5 shows a plot of a discriminant analysis similar to that in
Figure 1 with Winchester added as a candidate author.
Figure 5: Discriminant
Analysis, including Winchester. The Central America editorials are closer to
those of the Joseph Smith Group than to the Winchester texts.
The
first discriminant function (the dimension of greatest distinctiveness)
differentiates Winchester from the other three authors. The second function
differentiates Joseph Smith from Wilford Woodruff.
[Setting
aside the irony of Brother Roper resorting to this visual analysis after
criticizing me for doing the same thing, Figure 5 is quite revealing.
First,
Brother Roper is showing us he’s running a different analysis than he did in
his original paper; none of the three Figures from the original paper showed
the results that appear in Figure 5. His depiction of Discriminant Function 1
vs 2 in Figure 1 of this very paper is much different from what we have in
Figure 5. This leads me to wonder what the analysis would look like if he
simply plugged Winchester into the analysis he did in the original paper.
Second,
there are far more “Joseph Smith” data points in Figure 5 than in Figure 1,
suggesting that Brother Roper has converted some of the “Editor” text into
“Joseph Smith” text. This is particularly significant because in Figure 1, it
was the Editor texts that were closest to the “Central America” composite.
Third,
there are far more “Joseph Smith” data points than “Benjamin Winchester” data
points, and they are more spread out. Making the database for “Joseph Smith”
larger than for the other candidates makes the “Joseph Smith” circle much
larger as well. Had Brother Roper used the same number of data points for
“Joseph Smith” as he did for the other candidates, “Joseph Smith” would be a
smaller circle and may have been farther from “Central America” than
“Winchester,” depending on which data points Brother Roper retained. IOW,
Figure 5 shows how this type of analysis can be gamed to get the “right”
result.
Fourth,
even with this questionable and secret database, Winchester is closer to the
“Central America” text than either Taylor or Woodruff in both Discriminant
Functions. Of all four candidates, Winchester is the closest in Discriminant
Function 1.
Fifth,
Brother Roper offers only one “view” of the data in Figure 1 in this paper (Figure
6 in his original paper) because Figures 7 and 8 in his original paper showed
much greater distance between the “Central America” text and the “Smith” texts.
Now he’s showing only one view in Figure 5. Is it unreasonable to surmise this
is because the other two views also show greater distance between “Central
America” and Smith” like they did in his first paper?]
The Central America editorials clearly cluster with the Joseph
Smith Group and not with Winchester.
[This is a questionable claim for the reasons noted above, but
recall that in his original paper, Brother Roper commented on Figure 8 by
writing, “John Taylor’s texts are about as close on average as Joseph Smith’s
texts, indicating John Taylor’s possible influence in writing the texts signed
‘Editor.’” If we look at Discriminant Function 1 in Figure 5 above, the closest
data point is Winchester’s. None of Winchester’s data points are as far away
from Central America as about 1/3 of the Joseph Smith points. If Brother Roper
is consistent, he would have to conclude that this indicates Winchester’s
“possible influence in writing the texts” labeled “Central America.”]
Even Neville’s “eyeball test” would[Page
28]conclude that the Central America editorials are an “outlier”
relative to the Benjamin Winchester texts rather than the Joseph Smith Group of
texts. To make matters worse for Neville’s eyeball, the third function, which
is not shown in the two-dimensional plot in Figure 5, separates John Taylor
from the others and moves Winchester even further from the Central America
editorials.
Applying the appropriate statistical distance measure for
multivariate data — the Mahalanobis distance — the evidence shows the Central
America editorials to be an “outlier” from the Benjamin Winchester, John Taylor
and Wilford Woodruff groups, but not from the Joseph Smith Group. This is shown
in Table 3.
|
Joseph Smith
|
Benjamin Winchester
|
John Taylor
|
Wilford Woodruff
|
Critical Value
|
Mahalnobis Distance
|
15.97
|
25.48
|
45.61
|
54.35
|
16.27
|
Group Membership Probability
|
98.9%
|
1.1%
|
0.0%
|
0.0%
|
>95%
|
Table
3. Mahalanobis Distances and Probability of Group Membership for
the Central America Editorials. The Mahalanobis distance from the editorials to
the centroid of the Joseph Smith Group is not beyond the critical value. The
probability of the editorials’ membership in the Joseph Smith Group is 98.9%.
Since the Mahalanobis distance from the Central America
editorials to the centroid of the Joseph Smith Group (15.97) is not larger than
the critical value (16.27), while the distances to the centroids of the
Benjamin Winchester, John Taylor and Wilford Woodruff groups are larger than
the critical value, the editorials can be judged to be “outliers” from the
others, but not from Joseph Smith. Of the four, Joseph Smith is a much more
likely candidate as author of the Central America editorials. The probability
of group membership of the editorials with the Benjamin Winchester text is a
mere 1.1%. This shows even more lack of evidence contrary to the null
hypothesis.
[All of this is garbage in, garbage out, for the reasons listed
above. No amount of statistical manipulation can offset biased (or, worse,
secret) data.]
We could end the article here, since we have fully demonstrated
that Winchester is not a more likely author of the Central America editorials
than Joseph Smith. But, since Neville spends hundreds of pages trying to build
his case, it is necessary to analyze his methods more deeply. We will discuss
Neville’s “pseudo-stylometry,” and then to show further how poor a candidate
Winchester is, we compare him to an expanded pool of candidate authors
[The article might as well end here—the rest of the article gets
even worse. So far, Brother Roper has still not explained either his data or
his methodology.][Page 29]
Stylometry: The Statistical Analyses of Writing Style
Our
previous article and this article use stylometry: the
statistical analyses of written text to characterize the writing style of the
author. In authorship attribution it is necessary to first examine the
historical evidence for authorship. Without a solid historical foundation,
attribution assertions are baseless. Yet, if we consider only the historical
evidence, we can get only so far towards an answer, since multiple scenarios
could still remain plausible. Applying stylometrics, if done correctly, can
provide additional information showing who the most likely author is, given the
historical context.
Stylometry uses statistical measures to characterize an author’s
writing style and identify what makes it unique from other authors’ styles.
Many approaches have been used to define an author’s unique writing style. All
have looked at various features of an author’s writing as measures of style.
Some have counted letters, words, word-pair choices, unique words, word
lengths, sentence length, paragraph length, language complexity, and many other
metrics of style in an attempt to distinguish one author from another. Some
have proven to be better than others.
A good metric of writing style is one that is consistent within
an author’s writing and yet different from that of other authors. Many naïve
methods are not capable of meeting these requirements. Among these deficient
methods are average sentence length, unique words, and language complexity.
However, some methods are capable of satisfying the criteria.
Focusing on what are called noncontextual words has been shown
to be highly useful. Examples of noncontextual words are the words and, for, of,the, and to. These are function words — they do not convey the
author’s message but provide the structure by which the author forms his or her
message. They define the grammatical relationships among words instead of
conveying specific information themselves.
Noncontextual function words are used by all authors, but not in
the same way or with the same frequencies. Therefore, different usage
frequencies for noncontextual words are useful in characterizing an author’s
subconscious word “fingerprint,” sometimes referred to as his or her wordprint. Consequently, the use of noncontextual words
is a standard approach in the field of stylometry. Although the specific
noncontextual words that are distinguishing among authors vary from [Page
30]study to study, their effectiveness in measuring writing style
is well established.25
It is best to select the noncontextual words for a specific
study that truly distinguish the authors in that study.
[This is circular reasoning, as I’ll show soon.]
If this is not done, then the words selected may not be the ones
that will show differences among the writing styles in the study.
[IOW, if you don’t choose the noncontextual words that “truly
distinguish” the authors, then the words you select won’t show the differences.
This needs to be explained? Twice?]
Once the noncontextual words for a study have been selected, an
appropriate analysis method must be used. Discriminant analysis is well-suited
to the requirements for a good stylistic measure because it can find the
combination of weights for the words that (1) best shows consistency of
word-use within authors, and (2) at the same time discriminates among different
authors’ word-use tendencies.
Discriminant analysis is the method we used in our previous
article and one of the methods used in this article. Studies that do not use
powerful validated statistical methods are deficient and prone to yield
misleading, unsupportable conclusions. Neville’s methodological approach is a
textbook example of how not to do stylometric research and the consequences of
doing it inappropriately.
[I have to say, this is a ludicrous criticism. I don’t think much of
stylometry, especially Black Box stylometry, and I thought I made that clear in
the book. Brother Roper declined my request for his data and methodology; I
didn’t even attempt to guess what he did, other than to look at the meager data
he published in the form of his Figures. My “methodological approach” consisted
primarily of showing how Lund’s approach tended to support Winchester as
author. That wasn’t an endorsement of the approach, but an illustration of how,
even with that approach, Lund disproved Joseph was the author.]
Neville’s Pseudo-Stylometry
Noted
authorship attribution historian Harold Love says, “Anyone wishing to conduct
serious research in attribution studies cannot do so today without a good
general understanding of the nature and basic techniques of statistical
reasoning.”26 Neville, lacking such an
understanding, presents the unwary reader with several pseudo-stylometric analyses
which he claims provide evidence in support of his Winchester-authorship
theory. The main ones he uses are average sentence length measured as average
number of words per sentence, words unique to one author compared to those of
other candidate authors, and “cherry-picked” word-pattern similarities. All
these methods are amateurish, nondistinguishing techniques. We examine
Neville’s use of these in detail and show their deficiencies.
[This is
laughable. I explained that I am skeptical of stylometry generally, and I
assessed both Roper and Lund on this point. Here, I applied Lund’s methodology.
To the extent Roper agrees with me that Lund’s analysis is flawed, I’m fine
with that. But I wasn’t using Lund’s analysis to prove Winchester was the
author, only to prove that using Lund’s own methodology, Winchester was a
better fit.]
Average Sentence Length: Neville compares
Winchester’s average sentence length (ASL) to the ASL of the Central America
editorials, [Page 31]showing they are about the same (p. 225). However, ASL is not a
good measure of style by the two criteria for a good style metric: consistency
within an author and differentiation among authors.
ASL was first used to attribute possible authorship over one
hundred years ago.27 It is an archaic method that
has been shown to be unreliable and nondistinguishing. A specialist in disputed
authorship of documents, Patrick Joula, says, “Many other statistics have been
proposed and largely discarded, including average sentence length.”28 Naïvely using ASL can lead to
faulty conclusions and self-deception.
[So far, all of this is consistent with my critique of Lund and
stylometry generally. Brother Roper here is making a classic red herring
argument. This section of the paper represents an enormous waste of time by
Brother Roper.]
To show the deficiency of ASL to distinguish between authors we
use The Federalist Papers, commonly used for testing the
usefulness of authorship-attribution methods.29 Well-known stylometrician
David I. Holmes says, “The Federalist problem has
been used … as stylometry’s ‘testing ground’ for new techniques.”30
The authorship of twelve of the eighty-five Federalist Papers has been disputed, but
stylometric analyses have shown that they were all probably written by Madison,
with the possible exception of one paper. However, attempting to identify the
author of the disputed Federalist Papers using
ASL proves problematic. Figure 6 shows the ranges (lowest to highest) of ASLs
for the papers commonly attributed to Hamilton, Madison and Jay, along with the
range of ASLs for the disputed papers (with Winchester added as a comparative
control). In order to be comparable, Winchester’s [secret] texts
were concatenated and split into blocks of text that were about the length of
the average size of The Federalist Papers (2058
words).
Based on the ASLs, although Jay might be ruled out as the author
of the disputed papers, it would appear that Hamilton may be a better choice
than Madison, but the difference is small and unconvincing. Using ASLs as a
method of author identification fails to identify the author of the disputed Federalist Papers.[Page 32]
Figure 6: Ranges of Average
Sentence Length (ASL) for The Federalist Papers with
Winchester Added for Comparison. The ASLs vary widely within authors and do not
provide a basis to make convincing conclusions about authorship of the
disputed Federalist Papers. Winchester appears to
be the best choice as the author of the disputed papers — a clear sign that the
method is inadequate.
Furthermore,
ASL as a measure of writing style fails to distinguish the control author
(Winchester) from the other candidates. In fact, Winchester’s ASL matches the
range of ASLs of the disputed Federalist Papers more
closely than any of the actual Federalist Papers authors,
but Winchester had not yet been born when The Federalist Papers were
written. So using ASL can lead to absurd conclusions for The Federalist Papers. It is equally not useful when
Neville applies it to the Times and Seasons editorials.
Based on ASL, Winchester is more likely to be the author of the disputed Federalist Papers than he is to be author of the
Central America editorials.
[Not only
is Brother Roper making a red herring argument (he’s attacking Lund, not me),
but he’s displaying the fundamental problem with his own analysis. Without
historical context, stylometry has to include every potential author. Brother Roper’s
historical analysis is deeply flawed from every direction, starting with his
assumption that the author had to be residing in Nauvoo. There is an unlimited universe of potential
authors if one ignores the historical context.]
Words “Unique” to an Author: Neville focuses on
an author’s “unique” words, i.e., words which he claims one candidate author
uses but which the other candidate authors do not use. Such words are sometimes
referred to as “marker” words. However, stylometrician Leon Maurer notes, “It
turns out that rare words do not provide as reliable a ‘fingerprint’ because,
while it is easy to work in certain words now and then, it is hard to change
personal modes of common word use.”31
[This objection raises the issue of time disparity, which I suspect
I’ll return to shortly. But the “marker” words objection is a red herring, too,
as I’ll demonstrate.]
Again using The Federalist Papers as
a standard to evaluate Neville’s technique, we find that 16% of Madison’s
unique words are also in the disputed papers, 14% of Hamilton’s unique words
are in the disputed[Page 33]papers, and 5% of Jay’s unique words are in the disputed papers.
Adding Winchester to the mix as a control, we find that he has 3% of his unique
words (which is twenty “marker” words) in the disputed papers. Although 3% is
less than the percentages of Madison, Hamilton and Jay, Neville did not use
percent but only pointed out that there were some of Winchester’s unique words
in the Central America editorials. By his approach, we ought to conclude — as
Neville’s approach would require — that Winchester wrote the disputed Federalist Papers simply because the disputed
papers contain some of his unique words!
[This would be a useful observation if the authors shared with us
what “unique” words they used. Whatever secret words Brother Roper chose here,
evidently they are not actually unique; Brother Roper finds them in the papers
written by other authors. And, of course, the historical context is key, as
Brother Roper previously stated: “Without a solid historical foundation,
attribution assertions are baseless.”]
Repeating this exercise using Neville’s words in The Lost City of Zarahemla, we find that Neville
has forty-two unique words that appear in the disputed Federalist Papers. Neville’s method of pointing out
unique words used in the Central America editorials, and saying that that
provides evidence of authorship, would require him to conclude that he himself
had written the disputed Federalist Papers.
The use of unique so-called “marker” words fails to distinguish clearly the
author of the disputed papers, and Neville’s way of using them would not
eliminate the control, Winchester, nor would it even eliminate himself as the
author.
[This is the kind of argument that passed for “scholarly” in FARMS
and Maxwell Institute papers, so it’s no surprise to find it in the Interpreter. Note that I at least
explained my methodology; Brother Roper doesn’t.
At any rate, I understand why Brother Roper avoids the use of marker
words analysis, and I’ll demonstrate that below.
But two of the authors Brother Roper cites in this paper have used
“marker” words on the Federalist Papers with success. “Mosteller and Wallace
identified a list of 30 “marker” words, found to be more typical of either
Hamilton or Madison’s undisputed writings. They combined the evidence of how
often each of these words occurred in each disputed paper using Bayes’ rule,
and found that the odds on each paper having been written by Madison were at
least 240 to 1.” So contrary to what Brother Roper wants you to believe, using
marker words can be effective.]
So using Neville’s method leads to useless results. Thus
applying Neville’s “unique words” method is not reliable and cannot provide
useful support for his Winchester conjecture.
[Roper’s own citation says, “rare words do not provide as reliable a
‘fingerprint’ because, while it is easy to work in certain words now and then,
it is hard to change personal modes of common word use.” This is not the same
as “useless,” and the utility depends on the words and the proximity in time of
the sampled writings.]
Cherry-picked Word Pattern Similarities: Neville
singles out 73 words and phrases in the Central America editorials and asserts
that they are similar to words and phrases used by Winchester (pp. 207‒16).
[So far, this is 73 more words of detail than Brother Roper has
provided. Besides, he is assessing the First Edition of the book, which he
knows was preliminary—and intended to invite further research. The Second
Edition is far more detailed on this point.] However, his statements
rely only on mere circumstantial similarities. Many people use words similar to
the words in the Central America editorials simply because the words are
commonly used by English speakers.
[Well, a word such as “Zarahemla” is hardly “commonly used,” but it
was used 4 times in the Oct. 1 article. The word was used only 9 times in the
entire Volume 3 of the Times and Seasons
(24 issues). Once it was used in a reference to the title of Don Carlos
proposed newspaper; once in reference to a marriage that took place in Iowa;
three times in an article by Benjamin Winchester; and four times in the Oct. 1
Zarahemla article.
In fact, there are only two
articles in the history of the Times
and Seasons that refer to Zarahemla more than once (not counting an Iowa
conference report): Winchester’s article, and the Oct. 1 article.
It’s understandable that Brother Roper would dismiss such usage as
“common to English speakers,” and typically, he provides no evidence for his
assertion, but I’ll ask readers to think about just how common it was for
English speakers in 1842 to refer to Zarahemla multiple times in a single short
article. If Brother Roper told us what writing samples he used, we could see
how many contained this term. One word that never shows up in Joseph Smith’s
holographic writing, for example, is Zarahamla.]
At best the word pattern similarities only show that Winchester
and the author of the three editorials were speaking English.
[At best? This article is replete with this type of irrational,
counterfactual statements. I haven’t bothered highlighting all of them, but
this one in particular deserves a mention.
First, the word patterns show the author was writing English, not
speaking it.
Second, unlike the case with Joseph Smith, we have actual articles
written by Winchester on this specific topic. So the “word pattern
similarities” Brother Roper dismisses as irrelevant show a mutual interest in
this topic between Winchester and the anonymous author of the 900 words. The
similarities are also contemporary; Winchester wrote in 1841 and 1842, while
the 900 words were published in 1842.
The same cannot be said about Joseph Smith, despite Brother Roper’s
wishful thinking. There is not a single statement that can be unambiguously
tied to Joseph Smith that includes the words “Stephens,” “Central America,”
Guatemala,” or any of the other topic-specific terms from the 900 words. If
Joseph wrote these 900 words, then they are unique in all of his lifetime work.]
Contextually similar phrases are insufficient to attribute
authorship. Such similarities can only generate a question about authorship,
not an answer.
[This is a nonsensical statement. No one claims that Person A wrote
a particular article just because he has written about the same topic in the
past, but if he/she has never written
about the topic, and has never used
the terms associated with that topic, that person can be excluded as a plausible
author. Take the Federalist Papers as an example. Did the stylometrists
evaluating those papers check the writings of chemists or blacksmiths of the
day who had never written a word about politics? Of course not. Topical
relevance is sufficient to exclude entire categories of potential authors and,
if the topic is narrow enough, can point to a limited group of authors who have
written on the topic.]
As we have noted, noncontextual words rather than contextual
words are well-recognized among stylometricians as useful in characterizing
authorial styles, but all the words and phrases Neville focuses on are
contextual words.
[Noncontextual and contextual words are both useful tools. Brother
Roper excludes contextual words because they eliminate Joseph from contention,
not because they are not useful and probative.]
Contextual words are not as much an indication of an author’s
style as an indication of the author’s subject matter. None of the 73 words and
phrases that Neville focuses on is distinctive of Winchester’s writing style as
opposed to other authors’ styles.
[The list of authors in the 1842 time frame who wrote on this topic
is finite. First, they had to be LDS; no one else was trying to prove the
divine authenticity of the Book of Mormon. Second, they had to be writing about
Central America. That limits the universe to Benjamin Winchester, Orson and
Parley Pratt, John Page, William Smith, and W.W. Phelps. If there are others,
add them to the list. (Brother Roper likes to add Woodruff and Taylor, but
neither of them published anything on the topic. At most, Brother Roper can
claim they wrote anonymous articles, but that begs the question.) Third, they
had to be published in the Times and
Seasons. That excludes Page. It makes it very unlikely for the Pratt
brothers in 1842 for practical reasons. Plus, neither of them was known for
writing anonymously. That leaves only Winchester, William Smith, and Phelps as
plausible candidates based on style and contextual words.]
He tries to make Winchester’s use of the 73 words mean
something, when they are not distinguishing among other English-speaking authors
to begin with. [Page 34]His method is saying, in effect, “See, the author of the Central
America editorials uses these words and so does Winchester.” When in reality so
do many other English-speaking people.
[Brother Roper keeps repeating this point but the repetition doesn’t
help. If anything, he highlights the contrast between my analysis, which is
completely open and specific, and his analysis which is completely closed and
vague. I invite replication and collaboration; Brother Roper shuns it. On the
merits, though, what “other English-speaking authors” were writing about
Stephens and Zarahemla? Especially around 1842? If they exist, Brother Roper
needs to list them so we can discuss them.]
If we use Neville’s similarities between Winchester and the
Central America editorials and apply discriminate analysis with Joseph Smith
and John Taylor included, the evidence still indicates that Joseph Smith is
more likely to be the author of the editorials due to stronger similarities
than Winchester.
[This is sleight-of-hand. Brother Roper has expressly rejected a
contextual analysis. The discriminate analysis he purportedly conducted was
based on a secret database with all the problems I already mentioned.]
So we must conclude that there is no evidence that Winchester is
the author of the editorials based on Neville’s cherry-picked similarities.
[In the Second Edition, I go through each term and phrase in the 900
words. No cherry picking at all. I look at everything. That’s how I ended up
with Phelps’ participation.]
Conclusion about Neville’s Attempt at Stylometry: In sum, the
pseudo-stylometric analysis done by Neville is unreliable. He relies on average
sentence length, so-called “unique” marker words, and cherry-picked
similarities, all of which have been shown to be nondistinguishing.
[Anyone reading my first edition knows I’m skeptical of stylometry
and particularly “black box” work such as Brother Roper’s and crude methods
such as Brother Lund’s. Brother Roper’s own citation above contradicts his
assertion that unique words are non-distinguishing, and I cited one article by
authorities he cited to show that “marker” words are useful to determining
authorship.]
Neville’s “layman’s” observations, analytic methods and
reasoning result in unfounded, misleading, erroneous conclusions. They do not
provide valid support for his Winchester authorship theory. For further
consideration, Neville’s pseudo-stylometric analysis is evaluated in even
greater detail in the Appendix.
[Brother Roper keeps repeating his argument, but the repetition only
accentuates the fallacies of his own approach. I’ve shown how his historical
assumptions are unfounded and misleading. I’ve shown how his stylometry
analysis uses secret, questionable data with secret parameters and assumptions,
leading to a Black Box result. No matter how sophisticated the statistical
analysis, if the data is biased or irrelevant, the analysis is a failure. In
the next section, Brother Roper is going to demonstrate that his methods are
pure confirmation bias.]
Appropriate Stylometric Analysis
If
Benjamin Winchester should be considered as a candidate, perhaps there are
other early LDS writers who also should be considered. Up to this point all the
analyses we have shown use the data from our previous article. We transition
now to performing stylometry, using an expanded set of comparison authors.
To do stylometry appropriately one needs an appropriate set of
authors, focused texts, truly distinguishing features to analyze, and
high-powered methods to rule out unlikely candidates. Objective formal
scientific hypothesis testing methodology should be used.
How We Picked an Expanded Comparison Set of Authors: Although
we contend, on the basis of historical and statistical evidence, that the
Central America editorials authorship question is a closed-set problem, to
directly test Neville’s assertions we stylometrically evaluated Winchester as a
candidate author among an expanded comparison group of authors with historical
backgrounds that make them potentially plausible authors and who published
writings about American antiquities comparable in subject matter to the
unsigned Times and Seasons editorials during the same
period of time in Church history.[Page 35]
[Good. I welcome this.]
Expanded List of Other Potential Candidate Authors for the
Unsigned Editorials: In addition to Joseph Smith, John Taylor, and Wilford
Woodruff, it might be conjectured based on historical evidence alone, that the
unsigned editorials were written by other members of the Church who are known
to have written about the Book of Mormon and American antiquities previous to
the Times and Seasons editorials. These include George
J. Adams, John E. Page, W. W. Phelps, Orson Pratt, Parley P. Pratt, William
Smith, Erastus Snow, Charles B. Thompson, and Benjamin Winchester.
[Okay, but I’d like to know where Adams, Page, Snow and Thompson
were published in the Times and Seasons. Snow was published along with
Winchester. Articles about Adams were published (including two written by
Winchester). A review of Thompson’s book was published (probably written by
Winchester). But I’m happy to include all of them.]
Although ENSCM showed that Joseph Smith, John Taylor, and
Wilford Woodruff could be considered a closed set, we compared Winchester to
these others to see if he is the closest in style among them to the style in
the Central America editorials. If he is not, then his viability as a candidate
for authorship diminishes even more.
George J Adams. George J. Adams was an
actor who joined the Church in New York in early 1840. His flamboyance and
skills as an orator were often used in defense of the Church in the Eastern
States, and England, and reports [newspaper reports mailed to
Nauvoo, including four written under pseudonyms of Winchester and John Eaton] of his
debates with opponents were often printed in the Times and Seasons and the Millennial Star. He published several pamphlets in
defense of the Mormons.32 Adams visited Nauvoo in
September 1842. On September 7 the Prophet’s journal records:
Early this morning Elder Adams and brother Rogers from New York
visited president Joseph and brought several letters from some of the brethren
in that region. … In the P.M. brother Adams & Rogers came to visit him
again. They conversed upon the present persecution &c president Joseph in
the discourse to brothers Adams and Rogers shewed the many great interpositions
of the Almighty in his behalf not only during the present trouble, but more
especially during the persecution in Missouri &c. The remarks droped on
this occasion was truly encouraging and calculated to increase the confidence
of those present.33
[Page 36]The two men again visited the Prophet five days later on
September 12: “At home all day in company with brothers [George J.] Adams &
[David] Rogers, and councilling brother Adams to write a letter to the
Governor.”34The Prophet sat to have his
portrait painted by Rogers at his home on September 16, 17, 19 and 20.35 Following his excommunication
in 1845, Adams followed the leadership of James Strang, organized his own
church in 1861, and led an ill-fated attempt to settle in the Holy Land in
1866.36
[More evidence that they weren’t discussing Stephens and Central
America. Besides, I’m still waiting for evidence that Adams ever published
anything in the Times and Seasons.]
John E. Page. John E. Page, baptized in
1833 and ordained an apostle in 1838, received a call to accompany Orson Hyde
on a mission to Holy Land, but was unable to fulfill the assignment. He
actively labored as a missionary in the Eastern United States from 1840 until
1844, after which he rejected the leadership of Brigham Young and the Twelve
and became associated with several rival factions. Page visited Nauvoo for a
conference in April 1842, but then returned to Pittsburgh, where he resided
until June 1843. While there he published a short-lived newspaper, the Gospel Light, and two pamphlets in refutation of
the Spalding theory.37
[I’m still waiting for evidence that Page ever published anything in
the Times and Seasons.]
W. W. Phelps. W. W. Phelps had joined
the Church in 1831 and been the editor of the Church’s first newspaper,
the Evening and Morning Star, published in Independence,
Missouri, from 1832 to 1834. Phelps had written several brief editorials
discussing assorted reports of antiquities, including an article describing a
ruined city in Central America. He left the Church during the troubles in
Missouri in 1838 but returned and was rebaptized in Nauvoo during 1840. In 1843
he was considered, but passed over, for editor of the Nauvoo Neighbor, but there is evidence that Joseph
Smith made use of him as a ghost writer for some material attributed to the
Prophet during 1843 and 1844.38 It is conceivable that [Page
37]he may have contributed to or authored some of the articles published
during Joseph Smith’s tenure as editor.39
[Brother Roper forgot to mention that Phelps contributed to the Times and Seasons in 1841. I think he
also contributed in 1842. My analysis of the 900 words has led me to include
Phelps as a co-author and/or editor, as spelled out in the Second Edition.]
Orson Pratt. Orson Pratt, also an
early convert to the Church, was baptized in 1830 and became a well-known
missionary and writer. Like his older brother and fellow apostle Parley, Orson
labored in Great Britain from 1839 to 1841, after which he returned to Nauvoo
with other members of his quorum. In May 1842, he was out of harmony with
Joseph Smith and the Twelve over the issues relating to plural marriage. He
returned to full fellowship in early 1843.40
[“Out of harmony” in May? The relationship between his wife Sarah
and John Bennett was a key factor in the Bennett scandals that plagued Nauvoo
from July on. He went missing in July, possibly suicidal. In August, he was
replaced as a member of the Twelve. Orson Pratt is one of the least likely
plausible authors of the 900 words.]
Parley P. Pratt. Parley P. Pratt joined
the Church in 1830 after reading and gaining a testimony of the Book of Mormon.
His pamphlet A Voice of Warning was widely
read; and an 1839 expanded revision cited several reports of antiquities from
North and Central America which supported the Book of Mormon. He participated
in the apostolic mission to Great Britain and from 1840 until 1842 was editor
of the Latter-day Saints’ Millennial Star. Pratt returned to
Nauvoo in early 1843.41
[I’m still waiting for evidence that Parley ever published anything anonymously
in the Times and Seasons. However, it
is possible he mailed the articles from England.]
William Smith. William Smith, the
younger brother of Joseph Smith, was one of the earliest converts to the
Church. In 1835 he was ordained an apostle and continued to serve in that
office until the Prophet was killed in 1844. William’s relationship with Joseph
and fellow apostles from 1835 to 1844 was sometimes contentious. In April 1842
he became editor of The Wasp in
Nauvoo. In August he was elected a representative to the Illinois State
Legislature, but continued to edit The Wasp until
early December, after which he was replaced by John Taylor. Following the
martyrdom, he became Church Patriarch, but in later 1845 he broke with the
Twelve and was excommunicated; later he became associated with several
religious factions.42
[I think the evidence shows he was the actual editor of the Times
and Seasons from about May through October 1. The Wasp and the Times and Seasons
were printed in the same place and shared editorial content during this time
frame.]
Erastus Snow. Erastus Snow was baptized
in 1833. In the spring of 1840, at the suggestion of the Prophet he moved to
Pennsylvania, where he served as a missionary in Philadelphia, New York, New
Jersey, and [Page 38]Rhode Island. In September 1840 he returned briefly to Nauvoo to
escort his wife back to Pennsylvania, where he returned the following month. In
August 1841 he moved to Salem Massachusetts, where he labored until 1843. He
briefly visited Philadelphia in April 1842, after which he returned to Salem,
where he remained until his return to Nauvoo in March 1843. He would later
serve as an apostle from 1849 until his death in 1888.43
[I’m still waiting for evidence that Snow ever published anything on
his own in the Times and Seasons.]
Charles B. Thompson. Charles B. Thompson
joined the Church in 1835. After the Saints were expelled from Missouri,
Thompson moved to New York. In 1841 he published his book Evidences in Proof of the Book of Mormon in
Batavia, New York, and extracts from it were published in the Times and Seasons that same year.44 He moved to Macedonia in
Hancock County Illinois in the summer 1843. After the death of Joseph Smith he
formed a Church of his own and led a somewhat colorful career.45
[I’m still waiting for evidence that Thompson ever published
anything in the Times and Seasons.]
Benjamin Winchester. Benjamin Winchester, who
joined the Church in 1833, participated in Zion’s Camp in 1834. He published a
newspaper, the Gospel Reflector, in Philadelphia
from January 1841 to June 1841. From 1840 to 1843 he also published several
books and pamphlets. Winchester was an industrious writer and missionary, but
became a contentious figure during his time in Philadelphia from 1841 to 1843.
He returned to Nauvoo in October 1841, where he was reproved by Church leaders
for his conduct and counseled to do better. He briefly assisted as an editor of
the Times and Seasons from November until January
1841, when the Twelve, at Joseph Smith’s direction, purchased the paper from
Ebenezer Robinson. Winchester then returned to Philadelphia, where he continued
to cause problems in the local branch. In June 1842 he again visited Nauvoo for
a brief period, then returned again to Philadelphia until October of that year.[Page
39]He continued to cause difficulties in the Eastern branches of
the Church until he was excommunicated from the Church in 1844.46
[Brother Roper forgot to mention that Winchester had several
anonymous articles published in the Times
and Seasons in the 3rd Edition of the Times and Seasons, that he
had a long-time friendship with William Smith, that later in life he claimed he
was at the Times and Seasons until
John Taylor took over, etc.]
Although each of the above men had written on the Book of Mormon
and pre-Columbian antiquities previous to 1844, Page, Snow, Thompson, and
Winchester were not in Nauvoo during the fall of 1842, making them less likely
candidates as writers of the unsigned editorials.
[There is no factual reason why their absence makes them “less likely.”
Most of the material in the Times and
Seasons was mailed in. The 15 Sep issue includes an advertisement for
Winchester’s Concordance that had to
have been mailed in, making Winchester the one known person who submitted
something to the Times and Seasons for the 15 Sep issue.]
It is possible, however, that one of these men wrote the
unsigned articles and with the help of a collaborator in the Nauvoo printing
office may have succeeded in publishing them. Neville argues that Winchester
may have done so with the assistance of William Smith while Joseph Smith was in
hiding and unable to oversee the work in the printing office.
[No, not while he was in hiding and unable to oversee the office;
the evidence suggests that Joseph had ceased acting as editor long before
September.]
William himself may have written the unsigned editorials.
[Definitely possible, along with Phelps.]
The same could be said of George Adams, who met with Joseph
Smith in September 1842 and, given his interest in the Book of Mormon, could
conceivably have written or contributed to the editorials.
[This is implausible. Adams was an orator and actor, not a writer.
Theoretically, anyone could write anything, but as a matter of practicality and
probability, Adams is unlikely.]
Chronological considerations suggest that the Pratt brothers
likely did not write them. Parley, though familiar with Stephens’ work, was in
England in 1842. Unlike Parley, Orson was in Nauvoo in 1842, but was in the
middle of perhaps the most severe emotional and spiritual crisis of his life.
From May 1842 until January 1843 he was not involved in the work of the Twelve;
and with his faith and marriage in crisis, American antiquities and Book of
Mormon geography would likely have been the furthest topic from his mind.
As a journalist, Phelps could certainly write, was in Nauvoo at
the time, and given his activities as a ghostwriter for Joseph Smith, should
also be considered as a potential candidate. Although these candidates all seem
less likely [they “seem” that way to Roper, but not to history] than
Joseph Smith, John Taylor, or Wilford Woodruff, we have nevertheless included
them in our statistical analysis below.
The comparison set thus has nine authors as shown in Table 4.[Page
40]
Adams, George J.
Page, John D.
Phelps, W. W.
|
Pratt, Orson
Pratt, Parley P.
Smith, William
|
Snow, Erastus
Thompson, Charles
D.
Winchester,
Benjamin
|
Table
4: Expanded Set of Comparison Authors. There are nine in total
who can be considered plausible candidates due to possible historical
connections.
[Notice who
is missing? Joseph Smith, Wilford Woodruff, and John Taylor. Soon you will see
why.]
How We Selected Texts Specific to the Style of the Central America
Editorials: Efstathios Stamatatos, a specialist in textual analysis,
says, “Any good evaluation corpus for authorship attribution should be
controlled for genre and topic. … In addition, all the texts per author should
be written in the same period to avoid style changes over time.”47 To be able to distinguish
clearly between authors, we focused on constructing a study with texts from the
comparison group of authors that meet these three specifying criteria: (1)
genre matched, (2) topic matched, and (3) time period matched to the unsigned
Central America editorials.
Genre Matched: Since the Central America articles in
question are editorials, for genre matching we selected only published works of
an editorial or expository nature.
[Wait a minute. All the Joseph Smith data is holographic. Now Brother
Roper is excluding holographic
material from these alternative writers. So there’s a finger on the scale
already. That’s why Joseph Smith is not included in this list, but this is your
first red flag.]
This criterion is crucial because it is recognized that an
author’s writing style can change with genre.48 By focusing the text
selection on the editorial or expository genre we did not include items such as
personal letters, journal entries, or news items. If these other genres are
included in the analysis set they can dilute the accurate characterization of
the authors and confuse the results.
[Exactly! Brother Roper’s entire statistical analysis compares
Joseph’s holographic writings with published articles. That is in direct
contradiction to what he’s establishing here.]
Neville’s discussions refer to using a large corpus of articles
and other writings — an apparent potpourri of genres; thus he subjects his
conclusions to a multitude of potential confounding errors.
Topic Matched: The Central America editorials deal with
parallels between the recently explored Central America ruins and the Book of
Mormon. For topic matching, we selected only texts dealing with the relevant
topic as indicated by key words or phrases from the Central America editorials,
such as those shown in Table 5.[Page 41]
Antiquities*
|
Guatemala*
|
Narrow Neck of Land
|
Ruins
|
Central America
|
Incidents
|
Palenque
|
Stephens*
|
Darien
|
Isthmus
|
Quiriua
|
Zarahemla
|
Table
5. Typical Topic-Specifying Key Words in the Central America
Editorials. The asterisks indicate that we included all spelling variations.
[This is even more fascinating. I’m eager to see which of Joseph
Smith’s holographic writings include these terms. Except I know the answer:
none of them. Nor are there any writing samples from John Taylor or Wilford
Woodruff, apart from Woodruff’s journal that mentions Stephens. Now you see why
these three “candidates” are not included in this part of the study.
Let me emphasize this. Brother Roper’s criteria here exclude the only three people he tested in
his original article—and the first part of this article.]
These words are indicative of the topic covered by the unsigned
Central America editorials. We did not use “Book of Mormon,” since that phrase
is used many times in numerous articles that have nothing to do with Central
America. Some phrases, like “authenticity of the Book of Mormon,” were not
included, because they did not add any texts that were not already included by
those in Table 5. Other words that seemed peculiar to these editorials were
noted, but because they were not topic-specifying, they were not included.
[Okay, so before, context-specific words are not useful. According
to Roper, “so-called “unique” marker words… have been shown to be
nondistinguishing.” Now, only
“topic-specific” words can even be included in the analysis! If words that “seem
peculiar to these editorials” are in fact peculiar, then they are by definition
not “commonly used by English speakers” which was Roper’s objection a few
paragraphs ago.]
This topic criterion is crucial because it gives the best chance
of matching authors’ styles with the Central America editorials.
[If the topic criterion is actually crucial, which of Joseph’s
writings that Roper used fit this topic criterion? Which of Taylor’s or
Woodruff’s, for that matter? The only time Woodruff mentions Stephens is in his
journal, but Roper is now excluding journals. What we’re seeing here is Roper
switching horses. He’s gone from a Joseph Smith corpus of holographic writings
that never once mention Stephens or Central America, to a corpus by other
potential authors that excludes everything holographic and everything that
doesn’t mention Stephens or Central America.]
The inclusion of other topics has the effect of producing a less
focused style characterization.
[Except, apparently, if it is Joseph Smith. Then, the inclusion of
other topics focuses style
characterization. This is unbelievable.]
Neville includes a mix of topics in his textual analyses, thus
adding further confusion to his results and diminishing the distinctiveness of
the stylistic measures.
[I hope by now you can see the absurdity of this analysis.]
Time-Period Matched: The Central America
editorials were published in 1842. For time-period matching we restricted the
selected texts to those written from 1837 to 1852. [This is
arbitrary and introduces bias. Some of these candidates stopped writing about
this topic after 1844 or 1845. Others wrote about this topic before 1837. The
Stephens books weren’t even published until 1841. How about if Brother Roper
discloses his data and lets others play around with the dates?] Thus we
excluded texts written in the 1880s, for example. This criterion is crucial
because an author’s writing style can evolve over time.49 [The article
Brother Roper cites here compared writing styles of one author with a 30-40
year disparity and a second writer with a 27-year disparity. Here, Brother
Roper applies a 15-year disparity.]
[Presumably, Roper’s corpus of Joseph Smith’s writings are also
limited to 1837 to 1844; why not limit the others to that time frame?]
For example Sidney Rigdon’s writing style changed in his later
years from his early years in the Church.50
[I’m very interested to see what frontier authors didn’t change
writing style over 15 years.]
Again, if too large a timeframe is included in an analysis, an
author’s style in a relevant period can be diluted, and this can lead to
inconsistent results. In his analyses, Neville includes references to texts
from later time periods, which is methodologically unwise.
[I’m curious what Winchester texts are later than 1843. I don’t
recall him writing anything after that, apart from a couple of letters. William
Smith wrote some pertinent material through around 1846. I don’t recall using
anything as late as 1852, much less later than that.]
Twenty-one texts comprising over 114,000 words from the expanded
set of authors were found to match these three important specifying criteria.51
[Brother Roper’s footnote says “A list of
all of the texts used in the analyses is available from the authors to
interested researchers upon request.” I have been requesting a list
of his texts since January 2015, but so far he has declined to provide it. I’ll
consider this my latest public request. It would not have been difficult for
Brother Roper to list the texts here, or at least to provide a link. Instead,
he insists on screening the recipients of his data—a classic attribute of Black
Box scholarship.]
Note that we were careful to consider all the Winchester texts[Page
42]mentioned by Neville in his Appendices II and III. Most are off
topic. Those that are on topic were included in the analyses.
By focusing on the published editorial or expository genre, the
Central America ruins topic, and the relevant timeframe, we compiled a set of
texts that can specifically distinguish between the authors relative to the
Central America editorials. Without meeting these crucial criteria, analyses
can give misleading, erroneous results. Significantly, Neville’s naïve analyses
do not meet any of these criteria.
[Hmm, so Brother Roper has excluded Winchester’s March Gospel Reflector articles. Well, let’s
see how many of Brother Roper’s Smith, Woodruff and Taylor texts meet the
criteria. I have a sense this sentence is going to be the summary of my review
of this article: “Without meeting these crucial criteria, analyses can give
misleading, erroneous results.”]
How We Prepared the Texts for Analysis: To
guarantee that each and every word was correct, we independently verified our
electronic texts against photo copies of the original publications. If this is
not done, the computed frequencies of word usage can misrepresent each author
due to typographical errors. Neville did not verify all of the texts he used in
his analyses.
[Actually, I pointed out that the electronic texts have errors, but
so do different editions of the published texts. Every edition of Voice of Warning has significant
changes, for example.]
To get an unsullied characterization of the authors, we also
deleted all non-authorial words, like quoted material and scriptural
references. If this is not done, an author’s words can be mixed with the words
of other people and can once again lead to mischaracterization of his or her
style. Neville did not make this effort consistently in his analyses.
[Actually, I did, but oh well. But I think frequent quotations of
similar scriptural material is relevant.]
How We Found Truly Distinguishing Words: Since
all the authors share many words in common, it is necessary to find which words
are truly distinguishing. A criteria-based method of selecting words can be
used to provide a sound, unbiased basis for decisions. To obtain a set of truly
distinguishing words, we examined all words; only those that met the following
four criteria were selected:
1.
The word has to be a noncontextual word. This is a standard
approach in stylometry, as discussed previously. [So we’re using contextual
words to choose the database, and then we’re excluding them.]
2.
To help guarantee that the words used will differentiate among
authors, the word has to be one of the words whose range of proportions is in
the top five percent of all the words. A lower percentage gives too few words;
a higher percentage gives too many. [How can this determination
be made before we run the analysis? Let’s see the data on this.]
3.
To guarantee that the word is used frequently enough to give
statistically meaningful results, the overall pooled proportion for the word
has to be greater than one in a thousand. [There aren’t even 1,000 words
in the composite of the unsigned editorials.]
4.
To help guarantee statistical relevancy and ensure that the word
is characteristic of the author of the Central America editorials, the word
must appear at least three times in the composite Central America editorial
texts. [Are we talking about the 900 words or the five articles? As I’ve
pointed out, Brother Roper simply assumes all three (or five) editorials were
written by the same person. In fact, all are anonymous; they could have been
written by the same or different people. Plus, they all could have been edited.
At an average of 300 words each, it is questionable whether these articles are
even susceptible to statistical analysis. If a given word must appear at least
three times to “help guarantee statistical relevancy,” and the word doesn’t
appear at least three times in any one of the articles, then that word, by Brother
Roper’s own criteria, is invalid. Consequently, Brother Roper’s own analysis
can’t work except on his arbitrary assumption that one person wrote all there
articles! Not only that, the articles use the editorial we, which could be the
editor’s (whom we know was not Joseph Smith) or a combination of authors, or an
editor and author jointly.]
[Page
43]Thirty-seven noncontextual words met all four criteria: a, all, and, are, as, at, be, been, but, by, can, could, from,
has, have, his, in, is, it, more, not, of, on, our, so, such, that, the, they,
this, those, to, upon, was, we, will, with. Selecting words in this
fashion helps distinguish among authors using statistically significant words
specific to the Central America editorials. This methodological rigor
contributes to achieving the goal of high overall specificity for the study.
Validation of Word Selection Method and Discriminant Analysis: To
demonstrate the usefulness of our criteria-based word selection method and
discriminant analysis, we applied them to The Federalist Papers.
This yields seventy-five noncontextual distinguishing words. Figure 7 shows
discriminant analysis results for these words.
[This is a preposterously irrelevant comparison. None of the
Federalist Papers is as short as 300 words; there are only three possible
authors; most of the Papers were claimed by their authors; all the papers dealt
with the same subject matter, and so forth. Brother Roper here is invoking the
gold standard of stylometry to confer illusory legitimacy on his own analysis.]
The discriminant analysis had 99% correct classification of the
seventy training-set papers. Three papers are considered co-authored by Madison
and Hamilton, so they were not included. All but one of the disputed papers is
assigned to Madison — the attribution generally accepted by historians.
Figure 7: Discriminant
Analysis for The Federalist Papers. There are clear
separations among authors, and all but one of the disputed papers are assigned
to Madison, consistent with the findings of previous historical and stylometric
analyses.
[Page
44]Will this method also discriminate between The Federalist Papers authors and Winchester and
Neville? Including Winchester and Neville as negative controls in the
discriminant analysis generates the plot shown in Figure 8.
As they should, Winchester and Neville clearly separate from
Hamilton, Madison, and Jay, as well as from the disputed papers. Interestingly,
Neville’s style is the most distinctive, as captured in the first discriminate
function (horizontal axis). Winchester’s style, the next most distinctive, is
contrasted with Jay’s style in the second function. Although not shown in Figure
8, the third function displays less separation between Hamilton and Madison,
who are known to be similar in style. Thus we can see that the criteria-based,
word-selection method, coupled with discriminant analysis, form a powerful and
accurate technique.
[Of course, Brother Roper didn’t apply his exclusionary criteria to
this analysis, which makes one wonder what was the point of all the discussion
about genre, topic and time frame.]
Figure 8: Discriminant
Analysis of The Federalist Papers, including Winchester
and Neville. Winchester and Neville are easily distinguishable from The
Federalist Papers authors.
Objective,
Scientific Hypothesis Test Methodology
Having
observed that the three Central America editorials are unsigned and that
Neville offers Winchester as the author, we formulated the research question as
follows:[Page 45]
Is the writing style in the Central America editorials closer to
Benjamin Winchester than to the other candidate authors in the expanded set?
To test this research question we formulate the null (H0) and
alternative (Ha) hypotheses as follows:
H0:
Winchester’s style is not the closest to the style of the Central America
editorials among the other comparison authors (at least one other is closer).
Ha:
Winchester’s style is the closest to the style of the Central America
editorials among the other comparison authors.
Note that since we have already shown the results of an analysis
comparing Winchester with Joseph Smith, John Taylor and Wilford Woodruff, in
this analysis we compared only Winchester to the other comparison authors.
[Notice, Brother Roper can’t apply Joseph Smith, John Taylor, or
Wilford Woodruff here because they don’t satisfy the qualifying criteria. So
the first analysis is unreliable because of the secret data problems and the
second is unreliable because we still have secret data problems.]
Performing discriminant analysis, we obtained the plot of group
centroids shown in Figure 9. Six comparison authors are closer to the Central
America editorials than to Winchester. The probability that the Central America
editorials belong with the Winchester texts is less than one in a thousand
(< 0.001).
Figure 9: Group Centroids
from Discriminant Analysis. Winchester is not the closest to the Central
America editorials. Neither is William Smith. We point out William Smith
because Neville conjectures he could have been another possible source of the
editorials.
[Page
46]Robust Results: Many studies rely
on only one approach to analyze the styles of authors. But in order to not be
fooled by the results of only a single approach, we incorporated an array of
analysis techniques to confirm that the results are consistent and reliable.
[This is
another transparent garbage in, garbage out problem. It’s the choice and
manipulation of the data that is relevant, not the statistical manipulation of
the data.]
When
viewing the data from these various angles we can see a more robust picture of
the real situation. With the word-use proportions for the selected words for
each author, we performed the following analyses: Burrow’s Delta Method,
Discriminant Analysis, Fisher’s Combined Probability Test, n-Gram Matching, and
Principal Components Analysis.52,53
To help ensure that the results were not affected by the number
of texts we included for each author, we checked to see if there is any
relationship with sample size. There was no evidence of a strong relationship.
This indicated that the stylometric results are unaffected by varying sample
sizes.
[Seriously? We’re supposed to take their word for all of this? When
they have admitted all along that their objective was to show Winchester was
not the author?]
This array of five analytic techniques showed Winchester to be
an even worse candidate among a group of other plausible candidates for
authorship of the Central America editorials than when he was compared to
Joseph Smith, John Taylor, and Wilford Woodruff. The results are shown in Table
6.
[I don’t have any reason to question the statistical analysis here,
as a function of math. I do question the methodology associated with the data,
and my questions, so far, remain unanswered.]
Test Method
|
Number of Comparison Authors
Closer than Winchester to the Editorials
|
Principal
Components
|
8
|
Discriminant
Analysis
|
6
|
Burrow’s Delta
|
6
|
Fisher’s Method
|
2
|
n-Grams Matching
|
2
|
Table
6: Number of Comparison Authors Closer than Winchester to the
Central America Editorials. Among the expanded set of comparison authors, for
each test there are always other candidates who are closer to the Central
America editorials than Winchester.
Winchester is never the closest in any of these tests: Two
to eight other candidates are always closer in style to the Central America
editorials than Winchester. The highest he ever ranked was a distant third
place. Consequently, once again we find no persuasive evidence that [Page
47]Winchester is a good candidate for authorship of the three
unsigned Central America editorials.
[This outcome was predictable from the title of the article, so it’s
no surprise. But we still have no idea what Roper has done here with his secret
data.]
Stylometric Evidence Conclusion: The results of
multiple formal, statistical tests of hypothesis combined provide consistent,
overwhelming lack of evidence that Winchester is a viable candidate for
authorship of the unsigned Central America editorials.
[This is the same black box we’ve been seeing all along.]
Neville’s Highly Speculative Style
In
contrast to the evidence provided by these objective tests, Neville’s
conclusions throughout his book are not based on facts, but on a continual
framework of conjectures, speculations and suppositions — so much so that they
can be easily measured. He frequently uses speculative words such as could, maybe, perhaps, possibly, seems, suggests, supposedly, and a host of other similar words. Figure
10 shows a “word cloud” to illustrate how frequently he uses speculative words.
The most prominent word is suggests.
[This is
exactly what one does when one searches for the truth. One takes the historical
data—and I evaluated all the
historical data, not merely whatever matched my preconceptions the way Brother Roper
does—and proposed alternative explanations. Anyone can see the facts I’ve
assembled and can reach their own conclusions. That’s all I’ve done. In my
view, my proposal best fits all the historical data, including the
personalities and motivations of the people involved.
By
contrast, Brother Roper’s proposal that Joseph Smith wrote these 900 words
contradicts the historical data and what we know of the personalities and
motivations of the people involved.
All I do is
present the data. Anyone is free to reach the same or different conclusions.
What I
object to is Brother Roper’s long-established pattern of ignoring facts that
contradict his theories, inventing facts he can’t establish (such as everything
he thinks Joseph did just happened to be omitted from Joseph’s journals and all
extant letters, documents, and journals kept by others). Brother Roper’s theory
requires not only that we leave known facts unexplained, but that we assume a
long list of facts not in evidence that contradict known facts. My theory
explains every known fact and makes inferences about the unknown that are
consistent with what is known.
In this
article, Brother Roper has worked hard to come up with statistical analysis
that supports his theory. To do so, he has used a corpus of Joseph Smith’s
writings chosen by a set of criteria that he later completely disqualifies as
unreliable for other candidates. Then he acknowledges that he is keeping his
database secret.]
To see how unusually often he uses speculative words, we
compared them to the frequencies tabulated in the Corpus of Contemporary
American English (COCA), available online at http://corpus.byu.edu/. It is
described as follows.
COCA is the largest freely-available corpus of English, and the
only large and balanced corpus of American English. The corpus was created by
Mark Davies of Brigham Young University, and it is used by tens of thousands of
users every month — linguists, teachers, translators, and other researchers.
The corpus contains more than 450 million words of text and is equally divided
among spoken, fiction, popular magazines, newspapers, and academic texts. It
includes 20 million words each year from 1990‒2012 and the corpus is also
updated regularly. The most recent texts are from summer 2012. Because of its
design, it is perhaps the only corpus of English that is suitable for looking
at current, ongoing changes in the language.[Page 48]
Figure 10: Word Cloud of
Neville’s Speculative Words. The font size of each word is in proportion to how
frequently Neville uses that word in excess of common usage in American English
today.
Using
COCA, we calculated the difference in relative frequency of Neville’s use of
speculative words compared to their relative frequency in standard American
English today. Figure 11 shows the ten words with the largest differences.
Figure 11: Neville’s Top Ten
Speculative Words Compared to the Corpus of Contemporary American English.
Neville uses speculative words more frequently than standard American English.
We can
see that Neville uses these words in higher frequencies than commonly used in
Standard American English. For example, he uses suggests almost
1,000 times per million words more often in The Lost City of Zarahemla than
people use the word on average in a wide spectrum of texts.
[Seriously?
How about comparing my word usage to other historians?
By its nature, history requires us to draw inferences from limited facts. We make
those inferences and offer suggestions. IIRC, even a few archaeologists use
these terms… (See Mormon’s Codex, which uses these terms throughout.)
So I’ll
embrace this criticism. I’ve been explicit in my writing about what is factual
and what is an inference.
Earlier on
in his schizophrenic review, Brother Roper complained that I claimed everything
was a fact. Now he complains because I don’t.
I offer
suggestions and plausible explanations. Readers can easily see what is factual
in my book; I use footnotes extensively. By contrast, throughout this paper,
Brother Roper hides facts in favor of misleading his readers.
Throughout
my book, I invite additional research and perspectives. I propose alternative
interpretations of the data and welcome anyone’s opinion (although I prefer
that people rely on actual facts, instead of imagining what might have been,
heedless of the facts, the way Brother Roper consistently does.)
My approach
is in stark contrast to what we find in Brother Roper’s work, such as this
review. Instead of inviting additional research and analysis, he refuses to
disclose his data, methodology, and assumptions. Instead of even acknowledging
the many weaknesses of his approach, he claims infallibility and certainty.
That’s language
of confirmation bias, not legitimate research.]
Figure 12 shows cumulatively how frequently Neville uses
speculative wording in ten-page increments in the first 192 pages of his book.
In summary, Neville uses speculative wording over 800 times in the first 192
pages of his book. In one ten-page segment he uses an average of almost nine
speculative words per page. We can see that he starts off using speculative
words at a high rate, and then his rate of using [Page
49]speculative vocabulary increases as his narration continues.
From the information displayed in Figures 10, 11 and 12, we can describe
Neville’s style in The Lost City of Zarahemla as
“highly speculative.”
Figure 12: Frequency of
Neville’s Speculations. In the first 192 pages of his book, Neville uses
speculative words over 800 times, and his speculation rate even increases as he
goes along.
Neville’s
speculative language indicates the nonresearch nature of his work, since
speculative language is used more frequently in popular articles than in
research articles.54
[I trust readers
to make up their own minds based on the evidence I present. My approach is the
antithesis of Black Box scholarship.
Brother Roper,
by contrast, tells readers what he considers “fact” without giving them the
underlying data to make up their own minds. His results are not replicable and
his data is unknowable; mine can be accessed by anyone.
I’ll let you
decide which approach is more credible.]
Two
linguists who have studied speculative language and its functions, Elsa Pic and
Grégory Furmaniak, state, “If such hypotheses [speculations] were too numerous
in research articles, they would be severely received, as readers of [research
articles] are peers who do not accept unsupported conjectures and do not expect
to be treated as less knowledgeable.”55
Neville’s Speculations Unscientifically Morph into Facts
Within
a cloud of speculation, Neville is unable to distinguish fact from fiction. He
accuses Winchester of creating facts out of the whole-cloth of inference (123)
and disparages “Winchester’s inference … which morphed into a fact in his Times and Seasons articles” (p. 180). Yet he
himself does the same thing.
[Well, the 15
Sept. article in the Times and Seasons
does assert as fact something that is patently untrue. Roper wants us to
believe it was Joseph Smith who lied about the Book of Mormon. I think Joseph
knew the text better than that.]
Neville repeatedly creates “facts” morphed out of the
whole-cloth of his own original inferences, suppositions and speculations. For
example, on page 7 he speculates: “led me … to suspect someone else
entirely [Page 50]had written the 900 words.” Then one page later he asserts flat
out, with no hedging: “Joseph did not write these editorials.” Throughout his
book, there are numerous such morphs of speculated conjectures into statements
of fact.
Neville asserts as facts his speculations and spins a tale based
merely on things he imagines seeing in the data. If we use his speculative
vocabulary, The Lost City of Zarahemla “suggests, perhaps,
that maybe, it appears, that it could be that” his imagination is reality.
[Anyone can read the historical facts I’ve cited and see for
themselves. It’s an open model that invites further research and analysis. In
his reviews, Brother Roper hasn’t contested a single historical fact I’ve
cited—but he has ignored dozens, if not hundreds. Instead, he insists Joseph
did and wrote things for which there is not only no record, but there is no
precedent or antecedent. Everything Brother Roper asserts about Joseph Smith’s
authorship of these articles depends on the absence
of evidence—things not recorded in journals, letters, or other contemporary
accounts, things written anonymously, things written using words and phrases
Joseph never used at any time before or after, and so forth. Then, in a
last-ditch effort to prove with statistics what he can’t even plausible
establish with historical facts, he applies two phony stylometry analyses that,
in the first instance, applies special rules to Joseph Smith’s writings, and in
the second instance, excludes Joseph as even a possible author!]
Conclusion
Our
previous article, “Joseph Smith, The Times and Seasons,
and Central American Ruins,” concludes that our analysis pointed to Joseph
Smith as the most likely author of the Central America editorials, with
possible influence from John Taylor and Wilford Woodruff.
In The Lost City of Zarahemla Neville
conjectures that there is another author — Benjamin Winchester — and spins an
elaborate tale based on speculation and imagination which he often states as
fact while weaving a baseless story of conspiracy.
[It’s baseless only because Brother Roper doesn’t engage the
historical facts. I included hundreds of footnotes, mostly to original sources.
By contrast, Brother Roper offers 7 historical facts, all of which are
erroneous. I trust readers to decide whose theories are baseless. Brother Roper
compounds his misleading history with even more misleading “black-box”
statistical analysis of “black-box” data.]
He ignores the simple fact that unsigned editorials are common
in newspapers then and now, and do not imply a clandestine desire for anonymity
by the author. [Wow. It would be helpful if Brother Roper would do some research on
this point instead of making more bald assertions, but his main citation in this paper contradicts what he’s saying here. The Federalist Papers were all signed by a
single pseudonym. The authors didn’t take credit until much later in life. Brother
Roper would have us believe they did this because… why, exactly? Because they
wanted everyone to know who they were?
While that idea makes as much sense as most of Brother Roper’s
arguments, I suspect not even he believes that. There is a long history,
particularly in America, of desires for anonymity. In this specific case, of
all the potential authors for the 900 words, only Winchester had been formally silenced for all the Church to see.
Who of the candidates had a reason to remain anonymous other than Winchester?
That’s a question Brother Roper doesn’t even attempt to answer.]
In fact, the most logical assumption then and now is that the
editor is the author of unsigned editorials.
[This assumption is neither logical nor factual. Most of the
unsigned material in the Times and
Seasons consists of reprints from other periodicals with the author’s name
omitted, such as the many articles from Winchester’s Gospel Reflector. However, if Brother Roper’s statement here is a
concession that William Smith wrote the articles, I could accept that as a
starting point. Or maybe Brother Roper is back to insisting that Joseph Smith
sent himself a letter to publish, pretending he was someone else?]
It is also common practice now, as it was then, for editorials
to be the “voice” of the editor expressing the opinion of the publisher.
[This could use a citation or two.]
The most logical assumption is that editorials — signed or
unsigned — are official statements of the people responsible for the newspaper.
[This might be actually be a rational argument—the first one so far Brother
Roper has made—but it still begs the question of who was responsible, not for
the paper, but for writing and publishing these 900 words. The facts show
Joseph was not directly involved and probably didn’t go to the printing office
more than once a month. Of course, Brother Roper doesn’t address the facts; to
him, what the historical record does not say is more important than what it
does say. So it’s no wonder he relies on “logical assumptions” instead of
facts, but the lack of supporting facts render this argument invalid, after
all.]
When that is not the case, a disclaimer is published which says,
in effect, “The views expressed in this article are not necessarily the views
and opinions of the editor or publisher of this newspaper.”
[I’m curious about which frontier newspaper did this. Brother Roper’s
claim here is an anachronism comparable to the claim that the Stephens ruins
are actually Zarahemla.]
Further, he ignores the fact that it is completely irrational
for Joseph Smith to have published in the Times and Seasons three
editorials of unknown authorship that contradicted his views, since he took
over the editorship due to his concerns for what was being published in the
paper. And it would be even more irrational for him to publish material for
which he did not know the author after he had assured his readers at the onset
of his editorship that he was responsible for the content of the paper with the
clear statement “I stand for it.”
[This argument has been so completely debunked I won’t repeat that
again here.]
Even in the unlikely event that something he disagreed with had
slipped by his notice and was published three times, Joseph Smith still had
numerous opportunities and venues to correct those statements, [Page
51]even after he was editor. There is simply no logical basis for
Neville’s characterization of the publication of the unsigned Central America
editorials as being contrary to Joseph Smith’s views and due to clandestine
conspiracy. The Prophet could have corrected any errors at any time.
[I address this at length in the book. Joseph did take action.
Furthermore, the Zarahemla in Quirigua was never repeated. After Joseph died,
it was superseded by Zarahemla in South America.]
The first article in a series of three Interpreter articles showed that Winchester did
not promote a limited Mesoamerican geographical setting for the Book of Mormon,
but rather a hemispheric one. His ideas were nothing new and thus did not
warrant any subterfuge for their dissemination.
[If the 900 words were nothing new, why did Brother Roper write the
first stylometry article? Why did he write 55,000 words trying to attack my
summary and analysis of the history? I
give him credit for effort, but ultimately the desperation inherent in his
Black Box scholarship and inability to address the historical facts
demonstrates that his theory of what happened just doesn’t work.]
The second article showed that Joseph Smith was not opposed to
considering Central American cultural, geographical, and historical
correspondences with the Book of Mormon, but to the contrary found them
interesting and supportive of the Book of Mormon.
[The only historical evidence Brother Roper has provided of Joseph’s
“interest” in Stephens and Central America is the Bernhisel note, and that is
far more questionable than Brother Roper acknowledges. Otherwise, Brother
Roper’s arguments are a pile of inferences and suppositions, compiled into what
he thinks are “logical” conclusions. By contrast, I’ve marshalled abundant
historical evidence to support my proposals.]
In this third article we have shown the inadequacy of Neville’s
arguments. Neville says he sees an “outlier” in the discriminant plots in our
article “Joseph Smith, the Times and Seasons,
and Central American Ruins.” But statistical tests contradict his “eyeball”
test and show no evidence that the Central America editorials are inconsistent
in style with the texts in the Joseph Smith Group of texts. So the foundational
premise for his book is false. What he sees is due to his preconceived bias for
Winchester’s authorship of the unsigned Central America editorials.
[This is a funny Freudian slip by Brother Roper. It was impossible
for me to have had any “preconceived bias” because I had never heard of
Winchester until I did the research. I simply followed the evidence where it
led.
By contrast, Brother Roper has made a career out of promoting the
Mesoamerican theory. The secrecy and inconsistency of the stylometry approaches
he describes in this article reflect “preconceived bias” of the highest order. I’m more than willing to undertake a full,
open, objective examination of the stylometry evidence if/when Brother Roper is
equally willing to do so. So far, he’s refused. That says more than anything
this article purports to say.]
We have also shown that Winchester is no better candidate than
Joseph Smith as author of the Central American editorials; and we have further
shown, using an array of objective statistical techniques, that Winchester is a
poor choice among an expanded set of comparison authors. The historical and stylometric
evidence is overwhelmingly against Winchester as the author of the Central
America editorials.
[Brother Roper’s “expanded set of comparison authors” excludes Joseph Smith, Wilford Woodruff,
and John Taylor from consideration. By definition, that makes Winchester a more
likely candidate than any of them. I’m interested to see the underlying data.
We’ll see what Brother Roper is willing to share in the future.]
Neville’s book is at best a work of fiction. In fiction an
author can create an imaginary world to match the way he or she wants things to
be. However, in history and science we are constrained by the evidence provided
by data. There is only imagination in Neville’s pseudo-science masquerading as
history. The Lost City of Zarahemla is just the latest
entry Neville has added to the list of his other novels.
[There is admirable irony in this conclusion. Brother Roper’s entire
historical analysis consists of imagining what Joseph Smith could have done in
his spare time, unrecorded, and unremarked upon. His theory contradicts
everything else Joseph wrote or spoke about, as well as everything his
contemporaries wrote or spoke about what Joseph said and did. Brother Roper’s
theory ignores the real-world situation in the printing office in 1842, the personalities
involved, and the motivations of the people. Even historical novels are based
on some facts. In that sense, Brother Roper’s theory can’t even qualify as a
historical novel; it’s pure fantasy.]
Appendix:
Dissection of Neville’s Pseudo-Stylometric Statements in Appendix III of The
Lost City of Zarahemla
By his
own admission, Neville is an amateur when it comes to stylometry (p. 219).
Since he evinces no experience, expertise or sound judgment in stylometric
research, it is not surprising that he uses archaic, low-power,[Page
52]nondistinguishing methods, and jumps to baseless conclusions. In
the following we address by topic each of his assertions in Appendix III (pp.
217‒33) of The Lost City of Zarahemla.
[That’s
a nice string of adjectives, but it contradicts what I actually wrote in the
book. My skepticism of stylometry has been reinforced by the inconsistent
methods described in Brother Roper’s paper.]
Excessive Variation
• Neville says that a problem in applying stylometric analysis
is that the unsigned Times and Seasons editorials
vary so widely in style, content and approach that they cannot be grouped to
Joseph Smith (pp. 217‒218).
>> This statement is unfounded. Discriminant analysis shows
that the unsigned editorials group together and cluster with Joseph Smith’s
writings and editorials.
[Well, the analysis described in Brother Roper’s papers claims the five
articles had to be grouped together to have sufficient data for analysis. That
constitutes assuming the outcome; i.e., Brother Roper simply assumed the same
author wrote each one, and then sought to prove his thesis that Joseph was the
author by using different criteria for the Joseph corpus than he used for
everyone else.]
Outlier Claims
• Neville says that we have previously concluded that Joseph
Smith is the author “because his writing style is a little closer to the
unsigned articles than are the styles of Wilford Woodruff and John Taylor” (p.
218).
• He claims that these previous studies by Roper and Lund tend
to show that Joseph Smith is not the author of the unsigned editorials (p.
218).
• He repeatedly asserts that the Central America editorials are
“so distant from Joseph that it appears to be an outlier” (p. 219) and that the
composite of the Central America editorials “appears to be an outlier” (p.
220).
>> Univariate and multivariate distance measures show that
the Central America editorials are much closer to Joseph Smith than to John
Taylor or Wilford Woodruff. Multiple analyses testing for extreme values show
that the Central America editorials are not incompatible with the Joseph Smith
Group of texts. Neville’s opinions are not supported by objective statistical
analyses.
[Continually repeating the same mantra without addressing the
fundamental problems is not persuasive. The statistical analysis is not the
issue; it’s the validity of the data and the assumptions. We have five separate
articles, all by unknown authors. There is no principle of stylometry that
justifies combining them just because the researcher wants to have a single
author for each of them, or just because the researcher deems the articles,
individually, as too short for a legitimate analysis. No principle of
stylometry legitimizes the testing of one author’s holographic material, in
many cases over a decade old, involving topics and genres entirely different
from the other proposed authors’ corpus, against published and edited work by
the other authors and a composite of published and edited short pieces.]
“Someone Else Wrote Them”
• Neville says, “In my layman’s opinion, Roper’s results suggest
someone other than the three writers he tested actually wrote the 900 words”
(p. 219).
• He asserts that analyses by Roper and by Lund “assume” that
the only possible authors are Joseph Smith, Wilford [Page
53]Woodruff, and John Taylor; he ignores the role of William Smith
(p. 218).
• He claims that Roper made a “simple mistake, … forgot
about The Wasp,” i.e., about The
Wasp’s editor, William Smith (p. 220). [So
supposedly I “ignored” the role of William Smith but I focused on him? Nowhere
did Brother Roper consider William Smith in his original paper.]
>> Joseph Smith, John Taylor and Wilford Woodruff are the
three candidates for whom the historical evidence is the strongest, since they
were responsible for the paper and were known to be directly connected with the
Times and Seasons production during this time.
[But in his more detailed stylometry analysis, Brother Roper
excludes them as possible candidates! They don’t pass the genre, topic, or time
criteria.
Plus, most of the material published in the Times and Seasons was mailed in. Local authors were identified by
name (Eliza Snow, Phelps, Littlefield, and even Woodruff and Joseph Smith when
they actually wrote articles). There is no historical evidence to support the
idea that Joseph, John, and Wilford were candidates for anonymous articles, let
alone the strongest candidates. To the contrary; the historical evidence excludes all three as viable candidates
(short of assuming they wrote these articles months in advance, a proposition
that the articles themselves refute.)]
All other candidates are only circumstantially possibilities.
The Extended Nearest Shrunken Centroid Method (ENSCM) open-set test found no
evidence of a latent author, and thus no need to consider another candidate
besides the historically justifiable Joseph Smith, John Taylor and Wilford
Woodruff.
[This is nonsense. By Brother Roper’s own criteria, there is no
corpus for any of these three men that qualifies for analysis.]
Even so, when Benjamin Winchester and William Smith are included
individually as possible candidates along with Joseph Smith, John Taylor and
Wilford Woodruff, and when they are tested as part of the extended set of
comparison authors, statistical tests show repeated that neither is a likely
candidate.
[This is the sleight-of-hand trick. Joseph, John, and Wilford were
not—the could not have been—tested along with the extended set of comparison
authors. The test of Winchester against Joseph, John, and Wilford used a
different analysis than the original analysis, and the results were not fully
disclosed even then. Needless to say, Brother Roper never disclosed his data,
parameters, etc.]
In fact, when we took each author from the extended set of
authors (Table 4) and used discriminant analysis to compare his writings to
those of Joseph Smith, John Taylor and Wilford Woodruff, for nine out of nine
comparisons, the Central America editorials are closer in style to Joseph Smith
than the comparison author, and the lowest probability of group membership for
the editorials in the Joseph Smith Group is 99%. We did not forget anyone. No
one else is a more likely candidate than Joseph Smith. There is no evidence
that “someone else wrote them.”
[This is nothing but Black Box garbage in, garbage out.]
About Techniques
• Neville says, “A writer’s use of function words can be unique
enough to yield statistically significant results” (p. 217).
• He claims that collocation habits and rare pairs can be
distinguishing (p. 217).
• He claims, “I decided to apply similar stylometric
methodology” (p. 218).
>> Though Neville recognizes the value of function words[Page
54](noncontextual words), he does not use them in his analyses. We
use them in our analyses. Further, an author’s word pattern habits can be
distinguishing, yet Neville analyzed only a few collocation and word pairs,
whereas we analyzed all the phrases from two-word to six-word sequences for the
extended set of candidate authors in comparison to the Central American
editorials. We looked for the author with the greatest number of phrases in
common with the Central America editorials. As Figure 13 shows, Winchester was
never the top choice. The closest he ever came was third place — at best a
bronze medal but never a gold medal.
Figure 13: Comparison Authors
with More Two-word to Six-word Phrases in Common with the Central America
Editorials than Winchester. Winchester always shares fewer phrases with the
editorials than do other authors.
Neville
did not use valid and reliable stylometric techniques, so his claim of applying
“similar stylometric methodology” is a gross misrepresentation.
[It bears
repeating that Brother Roper excluded Joseph, John, and Wilford from his
extended analysis.]
Average Sentence Length
• Neville uses average sentence length (ASL) as an authorial
style metric (pp. 217, 225).
>> ASL is particularly weak and nondistinguishing as a
measure of authorial style. It is an antiquated and amateurish metric. The
following shows the deficiency of ASL as a stylometric measure:
Splitting each of the comparison author’s composite texts into
blocks that are roughly the size of the Central America editorials while
maintaining whole sentences, the ASLs of [Page 55]the
blocks are not consistent within an author, thus violating a crucial
criterion of a useful stylistic measure. The ASL of the Central America
editorials is 32. Figure 14 shows the range of ASLs of the comparison authors
and of the ASL of the Central America editorials.
Figure 14: Ranges of Average
Sentence Length (ASL) of Expanded Set of Comparison Authors and the ASL of the
Central America Editorials. The Central America editorials’ ASL is within the
range of ASLs for all comparison authors except William Smith. Clearly, ASL is
not a distinguishing measure.
Winchester’s
ASL is not distinguished from the other comparison authors’ ASLs. The range of
ASLs for each comparison author overlaps Winchester’s ASL range. All the
comparison authors’ ranges overlap each other. The ASL for the Central America
editorials is within the range of all the comparison authors’ ASLs, except
William Smith’s (who by Neville-logic would thus be disqualified as the
author). ASL is obviously a weak and nondistinguishing measure.
Skilled stylometricians abandoned using ASL a century ago.
Neville should also.
[This is an illusory complaint. I was responding to Brother Lund, on
his terms.]
Unique Words
• Neville discusses “unique” words or phrases he claims are
“exclusive to one author” (pp. 222‒25).
>> This is not a distinguishing metric, as we have shown
with The Federalist Papers example.
[Except the experts Brother Roper himself cited claimed the use of “marker”
words on the Federalist Papers was effective. Brother Roper’s academic
citations in the paper contradict his own assertions in more than just this
instance.]
In addition, each of the candidate authors in the extended comparison
group has so-called unique words compared to those of others. These range from
13% to 27% of their words, [Page 56]with
Winchester having 17% “unique” words. Seven of the other authors have more of
their unique words appearing in the Central America editorials than does
Winchester. Using Neville’s unique-words approach would actually disqualify
Winchester as the author of the unsigned editorials, since other authors are
better choices based on so-called unique words. Even Neville himself has some of
his unique words in common with the unsigned Central America editorials and, in
fact, more “unique” words than Winchester. Using Neville-logic, this is
evidence that he wrote the editorials. By his own method, Neville is a better
choice for author of the editorials than Winchester.
[Hopefully any reader can see how ridiculous and unscholarly this
paragraph is.]
Similarities
• Neville cites his Appendix II, where he annotated words in the
Central America editorials and notes that Winchester also used these words (p.
218).
• In particular, he focuses on three words: foregoing, credulous, andincontrovertible; and points out that Winchester also
used these words (pp. 221‒22).
• He discusses phrases offered by Lund such as assist us to, cannot doubt, cuts, the eyes of all the people,
and so forth. (p. 223).
• He focuses on several more phrases and words: none can hinder, so much, surely and great joy (pp.
225-26).
>> Neville bases much of his “analysis” on “cherry-picking”
similar wording and uses them to imply equality of source (same authorship).
His approach of searching for similarities is nothing more than snooping around
in the data looking for confirmatory evidence.
[Again, this was a response to Brother Lund’s approach.]
To see how absurd and misleading this can be, we applied his
method to his own book and looked for similarities between Winchester and
Neville. We found over fifty examples. Using Neville-logic, these similarities
between Neville and Winchester would mean that Neville and Winchester are the
same person, but such a conclusion is obviously absurd. To Neville, these
similarities would be crucial “facts” that prove equality, but such reasoning
is vacuous and intellectually dishonest.
[This is an outlandish claim, of course. The key in all of this, as Brother
Roper’s schizophrenic article admits at some points but disputes at others, is
historical context. I framed this as means, motive and opportunity. The lack of
these exclude most of the universe. Once all three are established, then one
can look at similarities. Here, Brother Roper’s irrational and inconsistent
stylometry analysis insists on Joseph Smith writing 900 words about Central
America and Stephens and the like for the first and only time in his entire
life, in a historical setting that makes it almost impossible for him to have
done so. Then Brother Roper excludes the one person who wrote specifically on
these topics and had the means, motive and opportunity to write the 900 words.
Brother Roper is so befuddled that he excludes Joseph as a possible candidate
for one of his own stylometry tests. So it’s not only the historical context
and authorial content that exclude Joseph: it’s Brother Roper’s own criteria!
Nevertheless, Brother Roper insists that only Joseph could be the
author.]
[Page 57]Although we can find similarities between two things or people,
similarity does not establish sameness or equality. Neville commits the fallacy
of equating Winchester with the author of the Central America editorials
because of “similarities” he thinks he sees. It is always possible to find any
number of superfluous similarities between two things, if we are determined
enough, but similarity does not establish equality.56
[Nice! A long-awaited reference to the Interpreter’s citation cartel—even if it is to support a mere
aphorism.
But think about this. Brother Roper can find zero similarities
between Joseph Smith’s entire writing corpus and the 900 words. Instead, he
manipulates both the corpus and the 900 words (actually, the five articles)
until he can find a discriminant that satisfies his desired outcome. But then
he’s caught in a bind because he realizes Joseph Smith doesn’t even qualify as
a candidate for his more “expanded” test. So instead, he focuses on one
stylometry test for one purpose, and another for another purpose. He ends up
with two Black Boxes that suit two different purposes—but neither one is
legitimate.]
To further illustrate the fallacy of this method, consider the
case of two identical twins. Many people have trouble telling them apart, since
there are hundreds of similarities in their physical characteristics, and even
in their personalities and behaviors. However, it takes only one feature to
tell them apart — perhaps one’s nose is a little different than the other’s
nose. Their myriad similarities do not make them the same person. We see, then,
that it is necessary to focus on distinguishing characteristics rather than on
similarities, or we risk being fooled.
Neville alters the phrase great joy and
then claims that since Winchester used the word joy a
number of times, his writing is similar to the writing in the Central America
editorials. Does such a similarity really identify him as the author of the
editorials? How many other people use the word joy? Millions!
Did you use it recently? If so, by Neville’s way of thinking, maybe you wrote
the Central America editorials. Nor are any of Neville’s other cherry-picked
similarities informative about the authorship of the editorials.
[Brother Roper’s twin example contradicts his point. First he claims
to have found Joseph Smith to be identical to the author of the 900 words—but then
he establishes criteria for the “expanded analysis” that excludes Joseph Smith as even a candidate. It’s as if the identical
twins are indistinguishable—except one is a zebra and the other is a dolphin.
Of course “joy” is not a highly distinguishing term, so Brother
Roper would focus on that. But what about Zarahemla? How many of the potential authors
used that term within the year?]
We put Neville’s “similarity words” to the test. The statistical
technique of stepwise discriminant analysis examines the groups within a data
set to determine the features within the data that are the most distinguishing
(discriminating) among the groups. It picks the most distinguishing feature
first and subsequent features in descending order of distinctiveness. Applying
stepwise discriminant analysis to the expanded set [Page
58]of candidate authors, we found that only 14 of Neville’s words
were even slightly distinguishing features among the authors. Therefore he is
correct that his words show similarity, but his word list also shows that all
the authors use those words similarly. This is depicted in Figure 15.
Since Winchester’s range of word-use frequencies for Neville’s
similarity words spans the range for those words in the Central American
editorials, he is “similar,” but all the authors’ ranges overlap with the range
in the editorials completely or mostly. So if Neville wants to conclude that
Winchester wrote the Central America editorials based on his “similarity
words,” he must also conclude that at least six of the other authors did so as
well, and maybe even the other two.
[While I disagree with what Brother Roper is saying here, I’ll take
it as a concession that Winchester is a viable candidate. Now, let’s look at
means, motive and opportunity. Anyone who does so will reach the same
conclusions that I have, although Phelps and Wm. Smith are developed more in my
Second Edition.]
Using Neville’s “similarities” approach, we could pick any one
of the nine authors and claim he was the author of the unsigned editorials.
[I’m sure Brother Roper, unconstrained by historical context and
facts as he is, could pick any of these. But of course Brother Roper doesn’t
mention that Joseph Smith doesn’t even
enter this race.]
Winchester is not a materially better choice than any of the
others. Incidentally, William Smith, whom Neville also suggests as the author
of the Central America editorials, is the least likely choice, since he has
least overlap of Neville’s similarities.
Figure 15: Ranges
of Word-Use Frequencies for Neville’s “Similarities” for the Expanded Set of
Comparison Authors and for the Central America Editorials. Winchester’s range
of word-use frequencies is “similar” to that in the Central America editorials,
since his range spans that of the editorials, but that is true for six of the
other authors as well.
At
best, similarities can only generate questions. In the case of Neville’s book,
it would be “Could Winchester be the author of the Central America editorials?”
Neville cannot validly [Page 59]assert that the supposed
similarities he claims to have found answer this question.
To quote Neville’s own words, “ Until now, these facts were
never put together. The various threads … woven here were loose strands,
unattached, unimportant, and unnoticed. Until now, they’ve been meaningless”
(p. 118). Despite Neville’s claims, they are still meaningless.
Other Specious Arguments
• Neville implies that if an article is unsigned, the author
“desired anonymity,” as if the real author had something to hide (p. 218).
• He suggests that someone wanting anonymity could alter their
style and include phrases borrowed from Joseph Smith to imitate the Prophet (p.
218).
[Actually, this is a well-established practice that bears further
research. See this article, for example.]
• To support his case, Neville cites an example of William Smith
borrowing wording from Don Carlos Smith (pp. 218, 229).
• Among his similarity arguments he asserts that “the proximity
of these uses over less than a year suggests a connection between Winchester
and these editorials” (p. 228).
>> Hundreds of unsigned articles were published in this time
period in myriads of periodicals. It is unjustifiable to conclude that all
anonymous authors were trying to hide something.
[No one is claiming anything of the sort.]
Rather, this was just simply part of common editorial practice
in those days, as it is today. Should we conclude that all the unsigned
editorials in Winchester’s own newspaper, Gospel Reflector, were
not signed because Winchester was trying not to reveal his identity in his own
publication and thus had something to hide? Why apply such a claim to
unsigned Times and Seasons editorials?
[The difference is evident in the 15 Sep and 1 Oct issues
themselves. They both contained significant letters written by Joseph Smith and
sent to the editor for publication. His authorship was highlighted and promoted.
The contrast with the unsigned editorials couldn’t be more stark.]
Winchester himself, in the Gospel Reflector,
borrows many phrases from others, as Roper has shown in the first article of
this series.
[Yes, but Brother Roper made some historical errors there, too.]
This is one of the reasons contextual words are not reliable as
distinguishing markers of authorship. Stylometric researcher John Hilton
stated, “Our wordprinting technique has shown that most highly skilled authors
(e.g., Twain, [Page 60]Johnson, Heinlein, etc.), when intentionally trying to imitate
the writings of different persons, are unable to successfully change their own
free-flow non-contextual word patterns enough to simulate a different
wordprint.”57
[Contra, read this. And highly skilled authors don’t typically “borrow
many phrases from others,” as Winchester and the other early writers did. More
important, I contend that these 900 words were edited, which confuses the situation
even more in terms of stylometry.]
The possibility that William Smith may have borrowed phrases is
irrelevant to the question of Winchester‘s authorship of the Central America
editorials. There is nothing informative about one author employing in his
writing useful phrases he or she may have found elsewhere.
[Seriously? So if an author borrows phrases from elsewhere, that has
no bearing on identifying the author? I can’t wait to read how stylometry
addresses that one. Especially when the writing sample is less than 300 words
and consists largely of borrowed phrases.]
Further, Neville’s “proximity argument” is a spurious assertion.
It is merely “guilt by association.” It is prima fascia [facie] obvious
that many things can be in proximity and not be connected. Neville relies only
on circumstantial evidence.
Neville’s Basic Conclusions
• He says, “In my view, the results of both analyses [by Roper
and by Lund] contradict the conclusions of their authors” (p. 218).
• He says, “Evidence suggests Winchester wrote these” (p. 227).
>> Neville’s view is based only on his preconceived bias
against the results of others’ research and his propensity to replace facts
with his imagination.
[This isn’t even clever rhetoric. First, it’s counterfactual; I
could have had no preconceived bias because I had never heard of Winchester
before I started investigating. It’s true that I thought Brother Roper’s data
contradicted his conclusion, but that’s because of his own graphics—interpreting
them visually the way he did in his first paper. I have replaced no facts; to
the contrary, I have incorporated every fact I could find. By contrast, I’ve
shown that all seven of the “facts” listed in Brother Roper’s initial article
were wrong or misleading. Instead of relying on facts, Brother Roper’s claims
fall in the “missing data” category; i.e., everything Brother Roper claims
Joseph Smith did and wrote was unobserved, unnoticed, and unrecorded.
Everything Joseph did and said that was observed, noticed, and recorded
contradicts Brother Roper’s position.]
Neville’s stylometric assertions are ill-informed and baseless.
None of Neville’s pseudo-stylometric statements are supported by the evidence
from appropriate analyses. He finds confirmatory evidence because all he is
looking for is confirmation of his theory. He seems to be in love with his
theory; and, like a love-struck suitor, everything he sees confirms his ardor.
The dispassionate, skeptical eyes of a historian, statistician and
stylometrician look at numerous objective statistical tests and see no
persuasive evidence that Winchester authored the unsigned Central America
editorials.
[Hmm, will the dispassionate, skeptical historian, statistician and
stylometrician please stand up? That description doesn’t fit the authors of
this so-called study.
Any readers who have made it this far should know that I commit to a
full, robust, and open stylometry analysis of Brother Roper’s data—assuming he
ever makes it available to me.
I have no dog in this race, apart from getting at the truth. That’s
how I started out and that’s why I approached Brother Roper in January 2014. So
far, as evident in this article, he has declined to collaborate in the pursuit
of truth. He has declined to open the Black Box for all to see what’s inside.
I think the reason is evident by the hash he’s made of the
stylometry.]
