Today, the stretch of Baker Highway between Scott High Drive and Old Jamestown Road might well be defined as the heart of Huntsville. It’s a bustling place, featuring Tennier Industries, the Grace Professional Centre, Scott Appalachian Industries and Heritage Hills Apartments.

But you don’t have to go back too many years to find a time when this was open farmland — a place where life happened for Flem Baird Duncan, Cassie Lee Duncan and their brood of 10 children. It was the start of a story that has since seen chapters written all over the United States.

Flem — also known as FB — and Cassie married on March 24, 1907, in the Brimstone community of Scott County. He was a carpenter, a veteran of the Spanish-American War; she was a school teacher. Together, they had 12 children, 10 of whom survived to adulthood, and began a family that has included a congressman and mayor of one of Tennessee’s largest cities, state lawmakers, a judge, numerous business owners and plenty of ordinary citizens who trace their roots back to Huntsville, Tenn.

No one knows exactly how many great-grandchildren, great-great-grandchildren and great-great-great-grandchildren Flem and Cassie have today, or exactly where all they’ve spread. Descendants maintain a family roll, but it’s constantly being updated as new babies are born into the family.

“I think the take-away,” says grandson Phillip Duncan, a journalist-turned-publicist who is elected to the city council in Falls Church, Va., “is two very average, very normal people — a teacher and a Spanish-American War veteran — got married in Scott County, Tenn. and created this family that wound up putting down roots all over the United States.”

And no matter how far away from Scott County they’re led by their careers and the goals they pursue, many of them, upon passing, have wound up right back where it all began — buried in the Duncan Family Cemetery that is on the old homeplace, overlooking Baker Highway just east of Scott High Drive.

This is the story of a boy from Paint Rock, a girl from Brimstone, and how a thriving family got its start.

Cassie Lee and Flem Baird Duncan married in March 1907 in Brimstone.

Duncan-Goad roots

Phillip Duncan is the son of Joe Duncan. One of two surviving children of Flem and Cassie, Joe Duncan is a veteran of the U.S. Air Force who was elected criminal court judge in Knox County and later presided over the Tennessee Court of Criminal Appeals. As a judge, he built a reputation as a firm, yet compassionate, jurist. Among his family, “Uncle Joe,” as he’s commonly called by even those who aren’t among his many nieces and nephews, is seen as a sort of patriarch of the ever-expanding Duncan family — the glue that has long held the clan together.

Speaking from his modest home in West Knoxville’s Suburban Hills neighborhood, Duncan recalls a moment from 62 years ago, at Flem and Cassie’s 50th wedding anniversary on the family farm in Huntsville.

“All 10 of us kids showed up back in Scott County for their 50th anniversary,” he said. “The yard was full of cars and everyone was laughing and having a great time. I was sitting on the porch with Papa and all of the family was out in the yard visiting. He says to me, ‘Joe, look out there in the yard. You see all those big fine automobiles out there and all my children?’ And his comment stayed with me to this day. He said, ‘If success is anywhere in the world, it’s out there in that yard. And I’m responsible for all of that.’”

Flem Baird Duncan was the sixth of 10 children born to Emanuel and Jane Cecil Duncan, a fourth-generation Scott Countian. His grandfather, Joshua Duncan, moved his family to Scott County in the 1830s, when it was still a part of western Campbell County. Flem was a devoutly religious man, a lay pastor of the old Helenwood Presbyterian Church, where he taught Sunday school for many years. 

Cassie Lee was the oldest daughter of William Fred Lee and Etta Tennessee Goad Lee. Her great-grandfather on her mother’s side, Joshua Goad, had settled near Huntsville with his cousins in 1803. Her father, William Lee, had come to Scott County from Livingston to teach school in the Brimstone community. He wound up serving as Scott County’s superintendent of schools in the 1880s, and Cassie herself taught public school for 40 years.

By the late 1800s, the Duncan family had moved from Paint Rock to the family farm in Huntsville — where the Tennier Industries textiles plant is now located. Flem Duncan grew up there, and many of his children were born there.

Flem and Cassie meet

Etta Lee died when Cassie was just six. After her death, William Lee left the school system and opened a dry goods store on Rugby Road — now S.R. 52 — with his late wife’s older brother, John R. Goad. That lasted until Goad was elected sheriff in 1894. He would later be elected as both circuit court clerk and trustee; Lee would go on to serve as postmaster at Glenmary.

Following her mother’s death, young Cassie was raised on her grandparents’ home in Brimstone. Joshua and Elizabeth Robbins Goad lived on what would later become the Lawrence Walker farm at Indian Fork. 

After becoming sheriff, John Goad had a house on the corner of the courthouse square in Huntsville, where the Baker law office is now located. Cassie would regularly visit her uncle and aunt, and that’s where she met Flem.

“Papa was attached with her right off,” Joe Duncan said. “He started riding his horse to Brimstone to see her. He’d go up and spend the weekend with the Goad family. And when she came down to visit Uncle John, Papa would go see her. They got married and that was the start of the family.”

A Helenwood start

After their marriage in 1907, Flem and Cassie Duncan settled down in Helenwood. Their property bordered the Treat School for Boys. Later, when they decided to move “from town to the country,” leaving Helenwood for the family farm where Tennier is now, the Duncan home became a dormitory for the Treat School, and was called the Duncan Dorm.

Flem and Cassie’s first children — twins Martha Inez and Etta Jane — were just days old when they died in 1908. They were buried in what would become the Duncan Family Cemetery on the family farm in Huntsville. Five more children were born in Helenwood — Marjorie, Dorothy, Fred, Frank and Fannye Helen. The youngest of the Helenwood children, Fannye Helen would turn out to be the only of the Duncan children to settle down in Scott County.

In the late 1910s, Emanuel and Jane Duncan deeded 13 acres — near where Heritage Hills Apartments are now — to Flem and Cassie, and they built their growing family a new home in Huntsville. There, five more children were born: John, Joe, Elizabeth, Martha Ann and Robert.

Finding a soulmate

In about 1930, an event happened that would help define the life of Joe Duncan, one of the youngest of the 10 children being raised by Flem and Cassie: the Phillips family moved next door to the Duncans in Huntsville. Most of the Duncan children moved away from Scott County before marrying. Joe was an exception, though no one would have known it in 1930, as the Great Depression was unfolding across the land.

It was that year that Mitchell Phillips — who ran the railroad tower at New River, where the U.S. Hwy. 27 underpass is now located — moved with his wife, Susie, and their children to Huntsville, next door to the Duncans. Joe was six at the time.

Fannye Helen, Joe’s sister, went down to meet the Phillips children.

“She came back and was telling Mama and Papa, ‘You oughta see these people down there,’” Joe said. “She said, ‘They’ve even got a boyfriend for me! They have Bud for me, and Elizabeth can have Virgil.’ They were gonna give Helen to John. Then she turned around and looked at me and said, ‘They’ve even got one for Joe! Lou is there and she’s five years old. They’ve even got one for Joe.’” 

Even at that age, Joe recalled with a laugh, “I thought, ‘I need to see what mine looks like.’ So the next time Fannye Helen went down I said, ‘You need to take me.’ She took me and introduced me to the Phillips family, and I thought Lou was the prettiest thing I ever saw.” 

Joe Duncan and Lou Etta Phillips would grow up together on the farm in Huntsville. He remembers that Lou’s sister, Lucille, once told her, “You better quit playing with that little Duncan boy; one of these days you’ll marry him.” And Lou said, “I wouldn’t marry that boy on a stack of Bibles!”

But fate had other plans.

F.B. and Cassie Duncan are pictured with one of their daughters, Fannye Helen Frazier. She was the only of the couple’s 10 children to settle down in Scott County.

Tornado strikes the home

On March 21, 1932, just a couple of years after the Phillips family moved to Huntsville, tragedy struck: a tornado destroyed the family home.

The Huntsville Tornado of 1932 was part of a greater storm system that killed more than 100 people throughout the Southeast. It was the first tornado ever recorded by the National Weather Service in Scott County. Thirteen people were injured.

Miraculously, none of the 11 people inside the Duncan home were hurt that night, though the home was leveled by the twister.

The storm struck at 9 p.m., just as Frank Duncan was returning home from a basketball game at Huntsville High School. Flem opened  the door to let his son in, and the wind was blowing so hard that trees were being uprooted. He quickly closed it and took cover under an organ just before the house collapsed.

Joe remembers sleeping through the storm. The posts on his old-fashioned bed may have saved him, preventing the roof from completely collapsing on top of him. He awoke to his father and brothers digging timbers off his bed. 

“When they got to me, my comment was to Frank, ‘Light the lamp so I can see how to get out of here,’” he said.

Electric wires were just being installed in Huntsville by the Tennessee Electric Power Company at the time. Joe remembers seeing the downed lines sparking on the wet ground as Flem Duncan lined up his kids to take them to the nearby home of Nannie Shoopman. As the family lined up to follow in their father’s footsteps so that they wouldn’t accidentally step on a live wire, Flem noticed that Fannye Helen was not in line.

“She was in  the eighth grade at Huntsville at the time, getting ready to graduate,” Joe said. “She was over in the rubble and she said, ‘I’m not leaving this place until I find my olympic blue dress.’ She had gotten it from Sears and it was going to be her dress for the graduation ceremony.”

Fannye Helen found her dress, and the Duncans found shelter in the Shoopman house. Later, they rebuilt their home with help from neighbors. The Byrd family lived just east — on what is now known as Byrd’s Curve entering Huntsville — and had a vacant home for the Duncans to live in while they built their new home.

“The people in Scott County were just beautiful people back then,” Joe said.

All of the Duncan children pitched in while the house was being rebuilt. Joe recalls idly passing the time by banging on a 2×6 timber with a hatchet. As he did so, he accidentally clipped the finger of his brother, John. Part of John’s pinky had to be amputated.

“John went to the legal fraternity at the University of Tennessee and to Congress and everywhere else with that stubby hand I had given him,” Joe said.

Death visits

In 1936, the matriarch of the Duncan family, grandmother Jane Cecil Duncan, died after falling ill. Though their twin sisters had died shortly after childbirth, most of the Duncan children had not experienced death until that time. It was a tragic event.

With telephone service not yet in existence, Joe — who was 12 at the time — was sent with his older brother John, 16, to the New River tower to telegraph news of his grandmother’s death to the rest of the family. It was a somber, three-mile trip. On the way back home, John took his younger brother’s hand. The two brothers were close throughout their lives. John would go on to serve as mayor of Knoxville during a period of racial strife, and later to Washington as a U.S. congressman, while Joe would become a criminal court judge. But, that day, they were just two brothers, walking hand-in-hand up New River Hill and mourning the loss of their grandmother.

Another tragedy

Just one year later, 7-year-old Vera Jo was killed. The daughter of Dorothy, the second-oldest of the Duncan children, Vera Jo was living at the farm while her mother worked in Cincinnati. 

The previous night, Vera Jo — who Joe said was more like a sister to him than a niece — had asked for his pencil. When he gave it to her, she hugged him and said, “Joe, I love you.”

As she was getting off the school bus at the family farm the next afternoon, a truck failed to stop. As it sped around the bus, it struck and killed Vera Jo.

“That was a traumatic event for our family because it was really the first death,” Joe said. “Grandma and Grandpa had died, but they were old people. Vera Jo was seven years old, getting ready to start growing up.”

Joe remembers the day of Vera Jo’s funeral. The kids were in the yard, crying. John, the older brother who would go on to become mayor of Knoxville, tried to console his siblings before he, too, finally sat down and began crying.

“I’ve never been able to forget the events that surrounded her there at the house growing up with us,” Joe said. “It’s hard to get those memories out of my head.”

F.B. and Cassie Duncan are pictured with their children and grandchildren at their 50th wedding anniversary. The picture was taken in front of the old Helenwood Presbyterian Church, where F.B. Duncan taught Sunday school for many years.

Spreading out

Fannye Helen went to Lincoln Memorial University on a basketball scholarship before working as a nurse at Fort Sanders in Knoxville and in Loudon, where she met her first husband, Charles “Lindy” Limburg. Following his death, she returned to Scott County and worked as a nurse for Dr. M.F. Frazier, who she later married. She was a nurse for Dr. Frazier until his death in 1968, and so became the only of the 10 Duncan children to settle down in Scott County. Her son, retired educator Duane Limburg, is the only of Flem and Cassie Duncan’s grandchildren still in Scott County. Fannye Helen’s two older daughters, Brenda Cecil and Linda Hogue, are now deceased.

The oldest of the Duncan children, Marjorie, decided to follow in the footsteps of her mother and maternal grandfather as an educator. She taught for a period of time at New River before going to business college at Chattanooga. There, she became a babysitter for a Jewish family and followed them to Cincinnati, where she married and settled down.

Dorothy followed her older sister to Cincinnati, taking a job at a department store and marrying a wealthy man, Fred Proctor, who owned a book-binding business.

Fred Duncan also wound up in Ohio, settling down in Dayton. He followed another Huntsville boy, who owned a barbershop in Dayton. There, he married a Catholic girl — noteworthy because, up to that point, there had been no Catholics in the Duncan family.

Frank Duncan was perhaps the wealthiest of the Duncan children. He joined the Civilian Conservation Corps after the tornado struck in 1932, and helped build many of the original trails in the Great Smoky Mountains National Park. He wound up in western North Carolina and established several businesses, including a Ford dealership and a motel. He was a prominent businessman in Franklin, N.C.

Martha Ann — who, besides Joe, is the last surviving Duncan child — moved to Cincinnati as well.

Robert dropped out of school at age 15 and left home, catching a ride to Cincinnati with a truck driver. He later got into the moving business and his son, Freddy, established Freddy Duncan & Sons Moving & Storage in Cookeville.

That left Joe and John — two lawyers who wound up making names for themselves in Knoxville.

From the farm to law school

One hot day while hoeing corn on the farm in Huntsville, John Duncan threw down his hoe and declared that he was leaving the farm. “He said, ‘I’m going to Knoxville, get an education, be mayor and then go to Congress,’” Joe later recalled.

Flem Duncan’s response? “Joe, you know he can’t do that by himself. You go with him.”

Later, after serving in World War II and graduating law school, the two brothers opened a law office in Knoxville. One of their first clients paid his $100 fee in ten dollar bills. John looked at the money, then at his brother, and said, “Joe, this sure beats hoeing corn, doesn’t it?”

John Duncan would indeed become mayor of Knoxville (see related story, Page B8) and was later elected to Congress, representing Tennessee’s 2nd Congressional District — which included Scott County — for 24 years until his death in 1988. His son, Jimmy Duncan, held his father’s congressional seat until retiring last year; his daughter, Becky Duncan Massey, is a state representative from Knoxville.

All these years later, Joe Duncan jokes that he helped his brother get started. John had been robbed, his wallet stolen, and he had no money. He asked Joe if Joe had any cash to spare, and Joe gave him $25.

“The next thing I know, he’s got all kinds of clients and he’s doing well. He never did pay me back that $25,” Joe said with a laugh.

Overseeing the family

It may not have been on a stack of Bibles, as Lou Etta Phillips declared as a young girl, but she did indeed marry Joe. They had gone their separate ways after high school — Lou working for the U.S. Census Bureau in Miami; Joe as a B-29 navigator in the Air Force, stationed in California. But they came together again in Scott County in 1946, at the close of World War II, and it wasn’t long before wedding bells were ringing.

“We had maintained contact, but I hadn’t heard from Lou in a long while,” Joe said. “I hadn’t written her and she hadn’t written me. I was on a B-29, which was a new airplane, getting ready to go to Okinawa after the Marines captured it. They dropped the atomic bomb and the war ended, and I didn’t have to go, but I was planning to stay in California and go to law school at UCLA after I got out of the Air Force.

“Then one day, out of the blue, Lou wrote. She said, ‘I’m going back to Scott County; what’s your plans?’ Well, I wasn’t planning to go back, but I knew that was an invitation to come and see her.”

When he took leave, Joe returned to Huntsville. He and Lou became reacquainted. 

“We decided we better make this pretty steady,” he said. The childhood sweethearts wound up marrying.

“I remember her statement when we were six years old: it would be a cold day in hell before she married that boy,” he said with a laugh. “So that was that.” 

Joe and Lou moved to New Jersey for several years while he worked for the FBI. Later, they returned to East Tennessee because John had decided to run for mayor of Knoxville and wanted Joe’s help. 

Back home, Joe Duncan made good on a promise to his mother, Cassie: he set about the task of cleaning up the family cemetery, which at that time was still just a rough cleared patch of land on the hill. His brother, Frank, who had found success as a businessman in North Carolina, commissioned the large Duncan stone that is visible from Baker Highway today, and had it set up in the cemetery. Passersby on the highway may notice that the stone isn’t lined up evenly with the roadway; instead, it sits at an angle. There was a reason for that.

“Papa wanted to sit on his porch and be able to look at that stone,” Joe said. So the stone was oriented to look across the road at Flem Duncan’s house. “He was so proud of that big Duncan name on that stone,” Joe said.

Nearly 11 acres of land from the original Duncan farm were set aside for the cemetery, and a nonprofit corporation was established for that purpose. Enlisting the help of his sister still in Scott County — Fannye Helen — and other members of the family, Joe Duncan set about the task of clearing the cemetery, purchasing headstones to replace the unmarked creek stones that had served as the original grave markers, and turning the plot of land into the cemetery that it is today.

Joe also established a family newsletter that was published regularly for many years, a family roll was developed, and the descendants of Flem and Cassie began meeting for a biannual reunion. The get-togethers have been held throughout the eastern U.S. — Cincinnati, Lexington, Asheville, Orlando, Gatlinburg, Knoxville . . . and, in the 1990s, when Tobe’s Motel was still suitable for large gatherings, the entire Duncan clan returned to its roots, coming back to Scott County as one family for the first time since Flem and Cassie’s golden anniversary celebration back in 1957.

Judge Joe Duncan (center) is pictured with his son, Phillip Duncan (right) and his nephew, Duane Limburg. Joe Duncan established a law practice in Knoxville and served two terms as Knox County’s criminal court judge before being appointed to the Tennessee Court of Criminal Appeals. Phillip Duncan is an elected city councilman in Falls Church, Va. Duane Limburg, a retired educator who served as principal at Capital Hill and Burchfield elementary schools, is the only of F.B. and Cassie’s surviving grandchildren in Scott County.

Time marches on

Things have changed tremendously in the 112 years since Flem Duncan and Cassie Lee were married. Back then, they moved from “town” in Helenwood to “the country” in Huntsville. Now, the town of Helenwood has faded from existence, and the Duncan and Phillips farms in the country have become one of the busiest parts of Scott County on weekdays — with hundreds of workers manufacturing sleep systems for the U.S. military at Tennier and hundreds more working at Scott Appalachian Industries. 

Eight of the 10 Duncan children have passed on, many of them buried in the Duncan Family Cemetery on the hill that overlooked the family farm. Lou, too, has passed. After a 35-year battle with cancer and 70 years of marriage to Joe, she died in the spring of 2017 and was buried in the family cemetery.

Throughout the United States, the great-grandchildren and great-great-grandchildren of Flem and Cassie Duncan continue to grow the legacy that began with a Spanish-American War veteran and school teacher on a small farm in Huntsville. Many of them have never been to Scott County and many never will. But because a boy from Paint Rock and a girl from Brimstone fell in love more than a century ago, what began in Scott County has established itself throughout the nation.

Local roots

From their start in Scott County, many of F.B. and Cassie Duncan’s children spread throughout the eastern U.S. John and Joe Duncan wound up in Knoxville, where John was mayor and congressman, and Joe was a criminal court judge. Their brother, Fred, landed in North Carolina and six of the other seven wound up in southern Ohio, with sister Fannye Helen being the lone exception. 

However, the family has many relatives still in Scott County.

F.B. Duncan was one of 10 children born to Emanuel and Jane Cecil Duncan. His grandparents were Henry and Catherine Thompson Duncan, and Samuel and Priscilla Martha Thomas Cecil. Cassie Lee Duncan was one of two children born to William Fred and Etta Tennessee Goad Lee. Her grandparents were Joshua and Elizabeth Robbins Goad. (Her paternal grandparents were from Overton County.)

 Among their modern-day relatives on the Duncan side are brothers Hubert and Ned Duncan, and their sisters, Raylene Terry, Geneeda Sexton and the late Delone Keeton and Marine Hall. Their grandfather, Ewell Duncan — father of Lonnie L. Duncan, who married Liza Jane Terry — was a brother to Emanuel Duncan, father of Flem Duncan. Ewell and Jane Terry’s other seven children included Florence, John Milton, William Ezra, Mandella Ruth, Ollie May, Silvester and Milford Elmer. 

On the Goad side, relatives include longtime educator and former Robbins Elementary School Principal James Walker and his brother, the late Lawrence Walker, who was a 32-year veteran of Scott County Commission (their siblings included brothers Kline, Stanley and Leon Walker, and sister Mary Helen Mosier). Their grandmother, Almira Goad, who married Daniel Walker, was the sister of Etta Tennessee Goad, mother of Cassie Lee. In addition to Thomas Walker and his wife, Zida Kline, other children of Daniel and Almira included John, William Franklin, Sarah Ettie, Vicy, Mary, Charlie, Flora, Houk, George, Minnie, Henry and Gilbert.