Royal Screwup on Day 2 of NYS Math Exam

Per upstate parent/teacher Chris Cerrone: BREAKING NEWS: “The 3rd Grade math test (Form D version) across the state contained blank pages with missing questions #33-37 & #43-48. Some schools caught the mistake and delayed the state of the test until copies of the missing questions could be obtained from BOCES were made. Other schools did not catch the error until the test began. These schools will be bringing their kids back to take the test once copies are made.

Another huge Pearson and NYSED fail.

The Third Grade test should be declared INVALID.”

The first news story to report this is here. Other reports have followed here and here and here.

According to page 14 of the NYSED testing guidelines (linked here to PDF): Testing materials can only be obtained from regional centers ON THE DAY THEY ARE TO BE ADMINISTERED.

So, replacement pages obtained today must be administered today. Not tomorrow or another day.

Also reports of some 5th grade tests pages not being cut (pages were connected and couldn’t be turned). Admin had to cut the pages open.

Additionally and unrelated, Wantagh High School, here on Long Island, was evacuated today as a result of a potential bomb threat. The neighboring middle school was also evacuated during the testing time. What are those kids supposed to do?

According to the NYSED testing guidelines page 25 (linked above): No student may be permitted to leave and then return to the testing room during any part of the test unless the student is accompanied by a proctor for the duration of his or her absence from the testing room.

So does that mean that the students who have seen the test today CANNOT come back tomorrow and take it because they will have left the testing room and returned to the same test?

Per page 43: Any breaches or irregularities, except interruptions caused by power outages or alarms, may be determined by the school or by the Department to invalidate a student’s test results.

This would include seeing the test, leaving the building, then returning to take the same test. Wording from page 43 seems to indicate students leaving for “alarm” does not invalidate the results.

 

Oh the glitches, those lovely glitches

So it appears there’s been some technical issues with the latest round of computer based testing. The question I will most certainly ask of my local school administrators when the time is right is: What will be considered tolerable? When 10% of students experience online testing “inconveniences?” Will it be 20%, 50%? The tests are high-stakes enough, then they have to go throw a Denial of Service attack on top of it, or random screen crashes, or back-end server failures, or whatever it will be next.

When these tests, across many states, are attached to such high stakes as grade promotion, teacher effectiveness, federal grant money, and more, these “glitches” are entirely unacceptable. Oh, and in many states, this is the second straight year of “glitches”. The phrase “Not Ready for Prime Time” comes to mind.

April 2, 2014: Kansas (here)

April 22, 2014: Oklahoma (here and here)

“A similar glitch stalled testing last year in Oklahoma, as well as in Indiana, Kentucky and Minnesota. Of those, Indiana also used CTB/McGraw-Hill as its testing vendor.”

April 23, 2014: Florida (here)

I am pretty certain we will probably be adding New York to this list next year or the subsequent year.

So now tell me how these test results can be used to measure anything of value? Or used to make any high-stakes decision? Not only can you not compare these tests to results of prior years of pencil/paper based testing, you can’t even begin to set the baseline for the new CBTs.

Here’s the recent response from Pearson: A Message Regarding Pearson’s Online Testing Services

The US federal government’s attempt at getting a web site ready to handle the crush of simultaneous visitors was very unsuccessful this past October. Do we think that the mad rush to get SBC or PARCC testing up and running will be any different. I don’t … and I really don’t think these testing consortium’s are heading any warnings nor learning from past non-education sector mistakes.

Related:
http://blogs.edweek.org/edweek/DigitalEducation/2014/04/oklahoma_students_face_second_.html
http://dianeravitch.net/2014/04/22/fairtest-computer-test-failures-are-common/

Pretty decent summary of the upcoming computer based testing.
http://www.edweek.org/ew/articles/2014/04/23/29cc-promises.h33.html

All I Can Do to Say Thanks

Thank you to my online friends and followers for putting up with my education posts and not all blocking me, unfriending me, or unfollowing me. At least, not yet.

Whether you had your kids refuse testing or not, or if you discussed it with family and friends or not, today was one of the greatest days in parent activism this nation has ever seen, led by knowledgeable and concerned NY parents.

Last year it was estimated between 1000-1200 refused the tests on Long Island out of nearly 215000 who could be tested. Today, we are over 15000 refusers and still counting out of 204000 possible test takers. My home district, where my son was only one of 3 in his whole building last year, topped out at 1398 refusers, 20% of test takers. His school had 32 this year. These numbers will go up in 3 weeks when the math exams are thrown at these kids. There were massive numbers in many districts. Numbers so high the tests are completely invalid and these kids are “More Than a Score.”

Consider this: no other nation over-tests their kids, especially their youngest and most vulnerable (those with special needs), every year from grades 3-12 with countless hours of high stakes exams. None. We stand alone in abusing these kids, hammering their confidence, and killing their creativity all so we can “evaluate their teacher”. Actually, so we can find a way of labeling them ineffective all because these kids could not sit for 540 minutes of high pressure exams. Almost every other nation simply tests students entering high school (or vocational school) and exiting. No, not us.

LI Principal Carol Burris, Education Historian Diane Ravitch, and CW Post Professor Arnold Dodge can sum up this movement much more eloquently than I can. Please read their thoughts here and here and here respectively.

So, one of the many points of this movement was not to coddle our kids, teach them to rebel against authority, nor shake up the schools this week. It was to send a message to Albany. A clear and loud message to NYS Education Commissioner John King and our Governor Andrew Cuomo: You refused to hear us for 2 years, now deal with us. And look what breaking news we have today. Low and behold…it’s our Governor with a “deal.”

Now, Cuomo plans Common Core changes for teacher evaluations

Oh and PS: just watched him on ch 5 news and he said the tests will not count for students and it will be good practice for them. I don’t even think he can make that decision. So, the kids now know it means $&@” and will not care how they do. How could he possibly allow that to be part of teacher evaluations. What a mess!!!!

All You Need to Refuse the NYS Elementary State Exams

Send the letter below via email and snail mail (hand delivered) to your child’s teacher(s) and principal. Refer to NYSAPE if you receive any push-back. Add in the Science exam if your child is in the 4th or 8th grade.

That is all.

Dear (name of administrator)

We are writing today to formally inform the district of our decision to refuse to allow our child (name) , to participate in the ELA and Math standardized assessments imposed on children across the state for the 2013-2014 school year. Our refusal should in no way reflect on the teachers, administration, or school board. This was not an easy decision for us, but we feel that we have no other choice. We simply see these tests as harmful, expensive, and a waste of time and valuable resources.

We believe in and trust our highly qualified and dedicated teachers and administration. We believe in the high quality of teaching and learning that occur in our child’s school. We hope our efforts will be understood in the context in which they are intended: to support the quality of instruction promoted by the school, and to advocate for what is best for all children. Our schools will not suffer when these tests are finally gone, they will flourish.

We do apologize in advance for the inconvenience or scrutiny that this decision may cause the administration, the school, and staff.

Sincerely,
(your name)

Thoughts on Data Protection Wording of NY Assembly Bill A8929

Here’s my take on the data collection/storage/sharing wording in the much discussed and debated NY Assembly bill A8929 that passed in the NY Assembly by a vote of 117-10 this past week. See the full bill here

I’ve copied out the data related sections and added my comments. I had to remove the ALL CAPS to make it easier to read (why is it in all caps like they typed it on a typewriter?)

_______________________________________________________

S  12. 1. Prior to July 1, 2015, the commissioner of education and the state education department are  hereby  prohibited  from  providing  any personally  identifiable information or de-identifiable student information to any third party vendor pursuant to any contract or memorandum of understanding for the purpose of collecting, storing  and/or  organizing student  data  or information in order to provide access to such data or information to third party vendors operating data dashboard solutions.

The wording above does not mean the state still can’t collect massive amounts of student, teacher, and parent personally identifiable information (PII). It just indicates NYSED can’t provide it to any third party for a year. Then, this gets revisited again next school year which simply prolongs this and we have to go through this all over again next year. The state can, and most likely still will, collect all the data it wants and will keep using it, just not work it into the new inBloom type systems. They won’t be able to provide the data dashboards to parents for at least another year. The wording below does not halt data collection.

_______________________________________________________

2. A parent of a student, a person in parental relation to a  student, or  a  student  eighteen  years  of  age  or older may request that such student’s personally  identifiable  information  and/or  such  student’s biometric  record  not  be  disclosed to any third party. The department and/or any school that receives such request shall  be  prohibited  from disclosing such information to any third party unless such disclosure is required  by law, pursuant to a court order or subpoena, for the purpose of a state or federal audit or evaluation to authorized  representatives of  entities  identified in section 99.31 (a)(3) of title 34 of the code of federal regulations implementing  the  family  education  rights  and privacy act, or is necessary due to a health or safety emergency.

The wording above indicates disclosure of PII might be required by law. So, spell it out. Which laws and when applicable?

_______________________________________________________

3. The department shall develop a form that shall be used for requests made pursuant to subdivision two of this section. Such form shall be made publicly available and shall allow such individuals the option to opt-out of disclosure of personally identifiable information and biometric records to any third party or to certain types of third parties. The department is authorized to identify a list of types of third Parties that individuals may opt-out of disclosure of such information and records and such individuals may opt-out of disclosure of such Information and records to any type and/or all of the listed third parties. Such list developed by the department shall not require the Names of such third parties to be listed. Such list may identify the Types of services such third parties provide.

This section above could cripple current school technology practices, which would be bad. The key phrase is “or to certain types of third parties.”  In modern-day 2014, we have to permit districts to use tech systems to facilitate transportation, scheduling, and other educational technologies including those used for instruction and learning by classroom teachers (Learning Management Systems, free/paid web based tools, cloud based email systems like Google Apps, etc.)

Wording also indicates NYSED will develop the list of services third party vendors can provide. The state doesn’t have to indicate the name of the vendor (e.g., inBloom) just what they do (e.g., data store). So NYSED can simply list out one of the possible services that parents cannot opt out from as “data store”, “data organization” or any other clever term used to continue their needs for collection, storage, analysis, mining, and sharing. Again, the prior paragraph indicates the storage tied to sharing is on hold until July 2, 2015, so NYSED can simply use next year to work on the list of services necessary and wait for July 2 hoping that no new legislation is brought forward to extend that temporary halt. This is also going to be a nightmare for schools to monitor and track what parent has opted out of what database/system/tool. The better approach is to simply eliminate PII from moving its way up the data levels to the ultimate spot of the NYSED offices.  Long term, I’d like to see a rework of the data levels 0, 1, 2, etc so PII never reaches the state level and never leaves the local BOCES offices. I still have not heard from NYSED why they need student/parent/teacher PII.

_______________________________________________________

5. Schools and the department may not under any circumstance  disclose personally  identifiable  information  or biometric records to any third party unless such third party has agreed in writing to:   a. Provide the department or the  contracting  school  with  a  breach remediation plan acceptable to the department or the school;   b.  Report  all  suspected  security  breaches  to  the  department or contracting school as soon as possible but not  later  than  forty-eight hours  after such suspected breach was known or would have been known by exercising reasonable due diligence; and   c. Report all actual security breaches to the department or  contracting  school  as  soon  as possible, but not later than twenty-four hours after such actual breach was known or would have been known by  exercising reasonable due diligence.

This section above is a good common sense approach. I would like to see added to that section details about performing security audits and making the results of such audits public. A state representative should be working with these third party vendors to verify that data is secure and not just take their word that it is.

_______________________________________________________
The major item I think is missing from not just this proposed bill, but any relating to data that I have seen to date, is what our fellow citizens in Oklahoma added to their new legislation on this topic last summer: an explanation of data fields. There bill isn’t perfect, but it better than what they had there. I want to see a law that mandates that ANY state department or entity that collects, stores, share, and uses data or is in contract with any third party vendor to collect, store, use, share  any PII related to the citizens of the state outline for citizens the specific types of data it collects (field names), the very specific purpose of the field (so in the NYSED databases they would have to explain in detail why every piece of data is needed), the length of time the data element is keep in the data base, if it is connected to any other state databases, and the specific details about any third party use of the data. Notice should be given to state citizens in much the same way the health care providers and insurance agencies have to notify customers of data use.

Go Ahead and Get the PARCC Testing Experience

Ok, you want to get a sense of what is coming in perhaps just two years in New York? Take a glimpse at the online based PARCC testing system developed by Pearson called TestNav8. As you go through the tutorial understand that 8/9 year olds will use this to take the spring state tests. The state put PARCC implementation on hold (NYSED’s favorite term) for one year. It is being field tested this spring and probably next fall and other states are already moving towards it.

In your mind think about taking these tests on pencil/paper and computer based (or tablet based with an external keyboard.) First thing that comes to mind for me: in school computer labs there are no kid sized keyboards. Typing anything of length is a challenge for them because of it and based on sample questions I’ve seen so far, there is a chance the students will be typing their long responses in a very small text editor where they can’t see the majority of their paragraphs on the screen. Add to this, the simple sound of keyboard tapping and we have an entirely new testing environment that was previoulsy devoid of outside sounds.

PARCC description of this tutorial: “This tutorial should be used to familiarize students with how to navigate the TestNav 8 computer-based environment (advancing, going back, tool bar, embedded supports and accommodations).”

Here is the site to check on a regular computer, not from a phone. The questions are not PARCC type questions. You can find some of those at this link.

Once again, we don’t even have short term studies from our state that indicate how students take tests differently on pencil/paper v. computer based testing (CBT). You’d need results from probably five years worth of tests that are half administered old way and half new way. Same test, different methods. Did it take more time on CBT or less? Were scores close to one another or not? What did students say about CBT? Teachers? Not just a one test, spring pilot. They are simply piloting this to check the tech end of things. The state will plow ahead with the “all in” approach instead of a gradual phase-in…again. I’d argue that the pencil/appear results from this year should not even be compared to future CBT.

Think about student erasing content/answers/their work. Easy in pencil world. On computer how many times does someone make a mistake in the beginning of a line and try to erase it only to erase the entire line. That could easily frustrate CBT’ers since they have a great chance at wiping out more work. The simple new distractions the online test brings to the screen are enough to distract a kid. There are two areas to work on math problems, turning ruler on and off, the reading guide on and off, changing color of the screen, and more.

Read more on the PARCC testing samples here at Chris Cerrone’s recent post on this topic. How about the potential for nealry 18 weeks of computer based testing under PARCC? Read more here.

The PARCC cost estimate is $29.50 per student, up from the current NY cost of around $15 per student (while I’ve seen a few posts online estimate the current NY costs, I have yet to find mention of it from NYSED.) The cost estimate does not include technology infrastructure and equipment upgrades necessary to deliver and administer the test. NY will be asked to come up with an estimated $30+ million more just to administer the test with districts having to come up with millions more to technologically prepare their schools. Also, PARCC is offering the test in paper-and-pencil format for an additional $4 per student in the first year.

Let’s hope PARCC remains permanently on hold. I want my kids to start using tech in schools to create, communicate, collaborate, not take tests.

No Need to Worry Your Children’s Data is Safe From the Feds

Hey, feds … nothing to see here. Move on.

The breaking ed-tech news last week (January 24, 2014) came in the form of this headline: State Chiefs to Arne Duncan: We Won’t Share Student Data

Upon first glance at the headline, I was overjoyed. However, within seconds it hit me. Actually two things hit me:
1. Why didn’t the state ed chiefs first send this letter, or their concerns, to their own constituents? You know, the folks they work for within their own states.
2. If the state ed chiefs felt the need to pen this letter to US Education Secretary Duncan, does it mean that there was (is) a chance that personally identifiable student information and test scores would be sent to the federal government? After all, if there was never a chance of that happening, there would be no need for this letter, correct?

The urgency of the letter stems from the development and potential implementation of two new standardized testing systems: PARCC and Smarter Balanced. These are two federally funded multistate consortia setup to basically develop and implement new computer/tablet based tests. Nealry $330million from the Race to the Top funds was granted to both organizations (see details here.) I’ve been warning local NY folks about PARCC for the last year for a variety of reasons (technology needs, cost, time constraints, data-mining, do we really want the only technology time our kids get in school to be testing time, etc.), but now it appears there is a chance the state ed chiefs are also “worried” about these new tests.

- What happens to this “agreement” if any of these state ed chiefs no longer hold their current positions?
– Exactly what data could have been shared with the feds if this agreement was not signed and sent?
– Would the student results of the tech based PARCC/Smarter Balanced tests be electronically sent to the feds?
– Why does the agreement between the US Department of Education and the testing consortia, PARCC and Smarter Balanced, have this clause (on PDF page 10 here):

The Grantee [the testing consortium] must provide timely and complete access to any and all data collected at the State level to ED [the US Dept. of Ed] or its designated program monitors, technical assistance providers, or researcher partners, and to GAO, and the auditors conducting the audit required by 34 CFR section 60.26.

- What school district has examined the NYS contract, or agreement, with PARCC? What is the district’s assessment of the agreement?

Please read more on the letter to Secretary Duncan and the data plans of PARCC/Smarter Balanced in the writeup from Leonie Haimson of Class Size Matters below.
What privacy protections are there when states share data with the testing consortia or with the feds?
http://nycpublicschoolparents.blogspot.com/2014/01/what-privacy-protections-are-there-when.html

Also note that just last week as well Indiana withdrew as a partner in PARCC. Will state partnership in this new testing project deteriorate like we have seen with the inBloom, Inc. project?

The Furor Over the NYSED Testing Link

I tend to be a pretty level-headed person and I tend to permit people to make mistakes and learn from them. I’ve been helping educators, students, and parents learn to use technology effectively and appropriately for nearly 20 years. Here’s my take on the current NYSED “bad link” scandal rocking social media today (see a complete roundup here).

I think the NYSED link to inappropriate content is a very unfortunate mistake. Most likely no one in NYSED has reviewed all the posted resource sites since first posting. But, they are today I’m sure. I seriously doubt this has anything to do with a hack, and it has NOTHING to do with inBloom. This was human error or someone’s attempt at a joke. One is excusable and one is not.

The quiz site NYSED linked to (http://www.allthetests.com/) is a Web 2.0 tool that can change at a moments notice. I can go make a quiz there right now about anything.  Anyone can see that. This is a site where you can both find AND create tests. Anyone at any time. That’s always the downside of pointing educators to links and online resources. They have to be carefully vetted to see if there is a publicly displayed “gallery” of some kind. When I used to vet sites for K-12, those would be ones I would not have used. Like Wikipedia…you visit a page today and it can change tomorrow.

As far as the ads on the quiz site, I have several web sites and if I suddenly decided to run ads on them, I can do so and add that “feature” to my site literally overnight. I might also not be able to control content in those ads depending on which ad service I use.  Their site has been using the same ad serving service for a number of years now so that doesn’t explain the “quality of the ads” we see today. Did the ad agency change the nature of the ads? Perhaps.

Additionally, I’ll add, NYSED should have a posted disclaimer somewhere that reads something like: “We are linking you to third party sites outside the control of NYSED officials. Please be aware that content can change and sites can be removed without notice. Please proceed with caution.” That’s what I had posted on pages when I setup “web resource” pages of links for educators. That disclaimer should be clearly visible on any page of NYSED that has many links that link outside the NYSED domain. Doesn’t excuse what happened, just adding some clarification.

Find the day/time NYSED added the link to their page, find the day/time those “sexy” quizzes et. al were added to the quiz site and I would speculate that the quizzes were not there when NYSED added the link for their page. If they were, they someone simply did a terrible job at vetting the site for educator/student use. That’s the root of the issue:  what did the allthetests.com site look like at the time the NYSED official added the link to the resource section?

I don’t think this goes to the top to King, nor do I think it is the Commissioner’s responsibility to be checking links. I dislike King’s handling of education in NY as much as anyone else, but I don’t think this is something worth using to try and bring him down. I honestly think this was just a really bad decision and not something malicious, or that the NYSED site was hacked, or event that King “did it” or should know about it. Whoever added the site is in trouble and whoever is in charge of periodically reviewing links is in trouble. I am probably going to take flack for this post and my explanation for the issue. So be it. Are we not allowed to make mistakes? In my opinion, this is really a non issue and certainly doesn’t warrant media attention. I’d be fine with NYSED officials just issuing an apology and saying they have removed the links and checked all links to outside sites found within the nysed.gov domain.

Here’s my challenge to all parents and educators: I can almost guarantee everyone that if you scour your own district, school, and classroom “web resource” pages where someone has posted lots of links for parents, etc. you will find at least one site linking to inappropriate content. I’d bet my next paycheck on it. No not really, but if I could afford it I would.

__________________

Followup: As I suspected, I really don’t think this testing site was “bad” at the time of posting to the NYSED resource page. Visit the Internet Archive and look for yourself using their Way Back Machine.   The first instance of this site containing the inappropriate links an categories, according to the archive is April 8, 2013. If the NYSED official added that link to the NYSED site before then, those nasty categories and tests did not exist. Again, this is the nature of trying to provide web resources for educators and parents and students; links change and need to be reviewed frequently.

Here is the site as of April 8, 2013 https://web.archive.org/web/20130408032437/http://www.allthetests.com/fun-tests-humor-quizzes.php

Here is the site as of March 2, 2013 https://web.archive.org/web/20130302232300/http://www.allthetests.com/fun-tests-humor-quizzes.php

The NYSED cached page that still has the direct link was last updated May 14, 2013. Any of the links on that page or the wording could have been changed that date and I am not sure how easy it is to remove a link from the cached version. I am sure that it is possible though. I can’t be bothered checking the archive for the page revisions to see exactly when the test link was added. For me, it’s been enough on this already.

Follow

Get every new post delivered to your Inbox.

Join 1,632 other followers